{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第一个线性感知机算法：PLA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 1. 什么是感知机（Perceptron）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**PLA**全称是Perceptron Linear Algorithm，即线性感知机算法，属于一种最简单的**感知机**（Perceptron）模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "感知机模型是机器学习二分类问题中的一个非常简单的模型。它的基本结构如下图所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../pic/1.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$x_i$是输入，$w_i$表示**权重系数**，$b$表示**偏移常数**。感知机的线性输出为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$scores=\\sum_i^Nw_ix_i+b$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了简化计算，通常我们将$b$作为**权重系数**的一个维度，即$w_0$。同时，将输入$x$扩展一个维度，为1。这样，上式简化为："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$scores=\\sum_i^{N+1}w_ix_i$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$scores$是感知机的输出，接下来就要对$scores$进行判断："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 若$scores\\geq0$，则$\\hat y=1$（正类）\n",
    "\n",
    "* 若$scores<0$，则$\\hat y=-1$（负类）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是线性感知机模型的基本概念，简单来说，它由**线性得分计算**和**阈值比较**两个过程组成，最后根据比较结果判断样本属于正类还是负类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. PLA理论解释"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于二分类问题，可以使用感知机模型来解决。PLA的基本原理就是**逐点修正**，首先在超平面上随意取一条分类面，统计分类错误的点；然后随机对某个错误点就行修正，即变换直线的位置，使该错误点得以修正；接着再随机选择一个错误点进行纠正，分类面不断变化，直到所有的点都完全分类正确了，就得到了最佳的分类面。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "利用二维平面例子来进行解释，第一种情况是错误地将正样本（y=1）分类为负样本（y=-1）。此时，$wx<0$，即$w$与$x$的夹角大于90度，分类线$l$的两侧。修正的方法是让夹角变小，修正$w$值，使二者位于直线同侧："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$w:=w+x=w+yx$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "修正过程示意图如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../pic/2.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "第二种情况是错误地将负样本（y=-1）分类为正样本（y=1）。此时，$wx>0$，即$w$与$x$的夹角小于90度，分类线$l$的同一侧。修正的方法是让夹角变大，修正$w$值，使二者位于直线两侧："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$w:=w-x=w+yx$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "修正过程示意图如下所示："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](img/../pic/3.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "经过两种情况分析，我们发现PLA每次$w$的更新表达式都是一样的：$w:=w+yx$。掌握了每次$w$的优化表达式，那么PLA就能不断地将所有错误的分类样本纠正并分类正确。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. 数据准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 导入数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集存放在'../data/'目录下，该数据集包含了100个样本，正负样本各50，特征维度为2。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv('./data/data1.csv', header=None)\n",
    "# 样本输入，维度（100，2）\n",
    "X = data.iloc[:,:2].values\n",
    "# 样本输出，维度（100，）\n",
    "y = data.iloc[:,2].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据分类与可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们在二维平面上绘出正负样本的分布情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEWCAYAAABliCz2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuUHGW57/Hvk0nC5Ea4JAoSkxE3\nXkJMhmSScMtFgQhe8IIiGBcHj+ywuQ6s4BEMkKgb9vHIMqAckIjHoztxi3I7yiKA6EYDBHXCRRA4\nwlaCObAliSRAhsBcnvNHdc90d3pmqnq6uqu6fp+1es1UdXXVW1XJM9XPU+9b5u6IiEjjG1HvBoiI\nSG0o4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGSEAr6knpl92cxurPayIdblZvYP1ViXSC2Y7sOX\nJDGz04HlwDuBV4DbgEvcfUc921WOmTlwiLs/W+a9+4DDgS7AgWeAnwKr3f2N4a5fpBK6wpfEMLPl\nwNeBLwITCQLmNOAXZjZ6gM+MrF0LIzvX3ScABxL8ETsFuNPMrL7NkqxSwJdEMLO9ga8A57n7Xe7e\n5e7PAScTBP3P5ZZbZWY3m9laM3sFOD03b23Buk4zs81mtt3MLjOz58zs2ILPr8393pJLy/wXM3ve\nzLaZ2YqC9cwzs41mtsPMXjSzawf6wzMYd9/l7vcBJwJHAB8eav1m9pvcxx8zs9fM7DNmtq+Z3WFm\nW83s5dzvU6K2R7JLAV+S4kigGbi1cKa7vwasB44rmP0x4GZgH2Bd4fJmNh24DlhKcGU9EThoiG0f\nDbwbOAa43Mzem5vfA1wITCII1McAZ0fcr8J9eR7oABYMtX53X5hbZpa7j3f3mwj+v36f4A/gVOB1\n4NpK2yPZo4AvSTEJ2Obu3WXeezH3ft5Gd7/d3Xvd/fWSZT8F/Nzd73f3N4HLCXLog/mKu7/u7o8B\njwGzANx9k7s/5O7duW8bNwCLou9akReA/SpZv7tvd/db3L3T3V8FrqhCeyRDkpz/lGzZBkwys5Fl\ngv6Buffz/jrIet5W+L67d5rZ9iG2/Z8Fv3cC4wHM7F3AN4E2YCzB/5dNQ6xrKAcBD1ayfjMbC6wG\njgf2zc2eYGZN7t4zzHZJBugKX5JiI/AG8MnCmWY2DjgB+GXB7MGu2F8E+vLaZjYG2L/CNl0PPE1w\np8zewJeBiguuZvZ2YA6wocL1LydIPc3PLZ9P+6gILKEo4EsiuPtOgqLtt83seDMbZWYtBLcybgH+\nNeSqbgY+amZH5gqgX6HygDiB4NbQ18zsPcBZlazEzMaa2SLg/wC/A+4Muf6/AQeXtOd1YIeZ7Qes\nrKQ9kl0K+JIY7v4/CK5yryIIhL8lSM8cE/bedXf/I3Ae8GOCq/1XgZcIvj1EdRHw2dw6vgvcFPHz\n15rZqwSB+2rgFuB4d+8Nuf5VwA9yd/GcnFvHGIL01kPAXVF3SLJNHa+koZnZeGAHQdrkL/Vuj0g9\n6QpfGo6ZfTSXRhlH8G3hceC5+rZKpP4U8KURfYzg9scXgEOAU1xfZUWU0hERyQpd4YuIZESiOl5N\nmjTJW1pa6t0MEZHU2LRp0zZ3nxxm2UQF/JaWFjo6OurdDBGR1DCzzWGXVUpHRCQjFPBFRDJCAV9E\nJCMSlcMvp6uriy1btrB79+56NyV1mpubmTJlCqNGjap3U0QkARIf8Lds2cKECRNoaWlBT4YLz93Z\nvn07W7Zs4R3veEe9myMiCZD4lM7u3bvZf//9FewjMjP2339/fTMSkT6JD/iAgn2FdNwa37p10NIC\nI0YEP9etG+oTkmWJT+mISHnr1sGyZdDZGUxv3hxMAyxdWr92SXKl4gq/3pqammhtbWXGjBl8+tOf\npjP/PyyCM844gyeffBKAK6+8sui9I488sirtlGxZsaI/2Od1dgbzRcpRwA9hzJgxPProozzxxBOM\nHj2a73znO5HXceONNzJ9+nRgz4D/4IMPVqWdki3PPx9tvkjDBfy4c5oLFizg2WefBeCb3/wmM2bM\nYMaMGVx99dUA7Nq1iw9/+MPMmjWLGTNmcNNNwUOMFi9eTEdHBxdffDGvv/46ra2tLM197x4/fjwA\nn/nMZ7jzzjv7tnX66adzyy230NPTwxe/+EXmzp3LzJkzueGGG6q7U5JKU6dGmy+Cu8f2InjoxOPA\no0DHUMvPmTPHSz355JN7zBvI2rXuY8e6Q/9r7Nhg/nCMGzfO3d27urr8xBNP9Ouuu847Ojp8xowZ\n/tprr/mrr77q06dP94cffthvvvlmP+OMM/o+u2PHDnd3X7Rokf/+978vWl/p+m+99VY/7bTT3N39\njTfe8ClTpnhnZ6ffcMMN/rWvfc3d3Xfv3u1z5szxP//5z6HaHuX4SbrE9e89C9audZ82zd0s+Jnm\nYxYmtuZftbjCf7+7t7p7W9wbiiunmb8ib2trY+rUqXzhC1/g/vvv5xOf+ATjxo1j/PjxfPKTn2TD\nhg28733v49577+VLX/oSGzZsYOLEiaG3c8IJJ/CrX/2KN954g/Xr17Nw4ULGjBnDPffcww9/+ENa\nW1uZP38+27dv55lnnhneTknqLV0Ka9bAtGlgFvxcs0YF26Hki92bNwd/JvPF7izc4dRQd+nEldPM\n5/AL+QAPjnnXu97Fpk2buPPOO7nkkktYsmQJl19+eajtNDc3s3jxYu6++25uuukmTj311L5tffvb\n3+aDH/zg8HZEGs7SpQrwUQ12YdjoxzLuK3wH7jGzTWa2rNwCZrbMzDrMrGPr1q3D2lgtc5oLFy7k\n9ttvp7Ozk127dnHbbbexYMECXnjhBcaOHcvnPvc5LrroIh5++OE9Pjtq1Ci6urrKrveUU07h+9//\nPhs2bOgL8B/84Ae5/vrr+z7zpz/9iV27dlV/p0QyIMvF7rgD/lHuPhs4ATjHzBaWLuDua9y9zd3b\nJk8ONYb/gK64AsaOLZ43dmwwv9pmz57N6aefzrx585g/fz5nnHEGhx12GI8//jjz5s2jtbWVK664\ngksvvXSPzy5btoyZM2f2FW0LLVmyhN/85jcce+yxjB49Gghu6Zw+fTqzZ89mxowZnHnmmXR3d1d/\np0QqkLbOX3FeGCb+WIRN9g/3BawCLhpsmeEWbd0bqxhTDSraSpzSWDiOq831OhYkoWhrZuPMbEL+\nd2AJ8ERc28tbuhSeew56e4OfjZ6TE6mnNHb+iqvYnYZjEWfR9q3AbbnxXEYCP3L3u2LcnojUWFrz\n4XEUu9NwLGIL+O7+Z2BWXOsXkfqbOjW4rbHc/KxJw7FouJ62ImmX+MJfgVreKFEvYc9HGo6FAr5I\ngqStU1Cjd/6Kcj7ScCzMB+hAVA9tbW3e0dFRNO+pp57ive99b51alH46funS0lI+LTBtWnATgtRW\nGs6HmW3ykCMZ6Ao/BDNj+fLlfdNXXXUVq1atqvp2NGyypKHwlyWNdj4aL+CXfmOpwjeYvfbai1tv\nvZVt27YNe12D0bDJohEwk6XRzkdjBfxVq+DCC/uDvHswPcyr8ZEjR7Js2TJWr169x3tbt27lpJNO\nYu7cucydO5cHHnigb/5xxx3H7NmzOfPMM5k2bVrfH4yPf/zjzJkzh0MPPZQ1a9YAaNhkAeIt/J19\nNowcGeSXR44MpqshTUXmvCQUYuty3ML20KrFa1g9bXt73dvbg+5t7e3lpys0btw437lzp0+bNs13\n7Njh3/jGN3zlypXu7n7qqaf6hg0b3N198+bN/p73vMfd3c855xy/8sor3d19/fr1DvjWrVvd3X37\n9u3u7t7Z2emHHnqob9u2rW87pdt1H96wyeppmz5x9BY/66ziHqD511lnDb+tjd7TNo7zUc3jRoSe\ntnUP8oWvYQ+tUBjk869hBnv3/sB72WWX+Ve/+tWigD958mSfNWtW3+ttb3ubv/LKKz5r1qyi4Lvv\nvvv2BfyVK1f6zJkzfebMmb733nv7xo0bi7ZTut3XX3/dp0yZ4rt37/bbb7/dP/vZz7q7+0knneSH\nHHJI37ZbWlr87rvvLlqHAr64uzc1lQ/4TU3DW++0aeXXO21aNVodjyS0uZptiBLwG2p4ZMxg9Wq4\n5pr+eatXB/Or4IILLmD27Nl8/vOf75vX29vLxo0bGTNmTNGywXnY03333ce9997Lxo0bGTt2LIsX\nL2b37t2DblfDJstw9fREmx9WGouaSWhzvdrQWDl8z+XsCxXm9Idpv/324+STT+Z73/te37wlS5Zw\n7bXX9k3nx80/+uij+clPfgLAPffcw8svvwzAzp072XfffRk7dixPP/00Dz30UN9nNWyyxKWpKdr8\nsNJY1ExCm+vVhsYJ+Plgf8010N4ejJ7W3h5MVzHoL1++vOhunW9961t0dHQwc+ZMpk+f3veA85Ur\nV3LPPfcwe/Zs1q9fz4EHHsiECRM4/vjj6e7uZubMmVx22WUcfvjhfevSsMkSl2Vln0Yx8PywklTU\nTEIhNqy6tSFs7qcWr2Hn8FeuLM7Z53P6uXx7Le3evdu7urrc3f3BBx/0WbNm1bwN7srhS2DtWveR\nI4vzxSNHVq8AWe+iZhIKsVFVqw1EyOE3Xk9b9+Kcfel0jTzzzDOcfPLJ9Pb2Mnr0aK677jrmzp1b\n83aop61AOnqMFora3rTtXzVF6WnbWEVb2DO41yHYAxxyyCE88sgjddm2SKkkFCqjiNretO1fvaQi\nh5+kbyFpouMmeUkoVEYRtb1p2796SXzAb25uZvv27QpeEbk727dvp7m5ud5Nqbq4eihGWW9SepeG\n7T0btUhY72Mctb1JKMSmQeJTOlOmTGHLli1s3bq13k1JnebmZqZMmVLvZlRVfrja/KPk8sPVwvCG\noY2y3rjaENXZZ8P11/dP9/T0T193XfGy+XatWBGkOaZODYJhufYm4RhHaW8ly2dV4ou2IoXiKs5F\nWW9SCoQjR5bvONXUBMO5OzcJx1jCi1K0VcCXVBkxonyXCrOg60Ut1htXG6Ia7H6E4fy3TsIxlvA0\nHr40rKjFubA54yjrTUqBMCm9Z+M4xpVIYw2m5sLesF+LV7mOVyKFonSwScKycUrCCJhJOW5JaUc9\n0EijZYqUCttDMeqIhFF6Piahp6Z7ENzzI2E2NQ0/2Ocl4RhHEaUdSRgts5qiBHzl8KVhKWccv6Qc\n4zTWYKpFOXwRkpNrb2RJOcZprMHUgwK+NCx1xqlcXB2k4iqWRmlHpv9dhM391OKlHL5UW1Jy7WkS\n18iTcRdL01iDqQaUwxeRSqnjVboohy8iFYtr5EmNaFl/CvgiUiSuomaWi6VJoYAvkhOloBh2lMq4\n2xGHuIqamS6WJkXYZH8tXiraSr1EKSjG1cM1ajviFFdRs5GKpUmBirYi0UQpKMY1SmXUdoiAirYi\nkUUpKJYL9oPNj6sdIlEp4IsQraAY1yiVUdshEpUCvghB4XD06OJ5o0eXLyjmn9IUdn6UIqwKmxIn\nBXyRnNJy1kDlraOOCvL4hUaODOaXyj/Wb/PmYH35x/oNFPSXLoU1a4KcvVnwc80aPapPqkNFWxHi\ne8ShirASNxVtRSKKUiyNa1mRuMUe8M2sycweMbM74t6WSKmwHaTiGl5Xj/WTJKnFFX478FQNtiNS\n5Oyz4frr+2+X7OkJpssF/biG1/3Qh8q3baD5UUSpD0StJUiDCttDq5IXMAX4JfAB4I6hlldPW6mm\n/KP/Sl9NTeWXj2N43Tgfp5flx/pJP5LS09bMbgb+BZgAXOTuHymzzDJgGcDUqVPnbC5X4RKpgNnA\n79XqXoU4H6eX5cf6Sb9EFG3N7CPAS+6+abDl3H2Nu7e5e9vkyZPjao5kUJwdpMKKM4efpFqCpEOc\nOfyjgBPN7Dngx8AHzGxtjNuTBElCgTDODlJhRenQVcm69Vg/iSRs7mc4L2AxyuFnRlJGfHQPRrDM\n5/KbmgYe0TKuNq9d6z5qVPF6R43SY/2kekhKDj/PzBYzQA6/kDpeNYY0djbSY/0kraLk8NXTVqou\njQXCuNqcxmMh6ZKIoq1kVxoLhHqsn2SBAr5UXZIKhGELsXqsn2RC2GR/LV4q2jaOJBQIoxZi9Vg/\nSSOSVrQNSzl8qSYVTCULlMMXQSNVipRSwJeGpYKpSDEFfGlYV1wBo0YVzxs1SgVTyS4FfGlopQOo\nDTagmkijU8CXhrViBbz5ZvG8N98M5otkkQK+NCwVbUWKKeBLw4patE3CCJ8icVLAl4YVpZerHgEo\nWaCALw1r6VJYsyboaGUW/FyzJphfasUK6OwsntfZqXy/NBb1tBVBo1pKeqmnrUhE6qQlWaCAn3Eq\nVAaijmqp4yZpNLLeDZD6yRcq87nrfKESyue5G1l+f1esCG7bnDo1CPbljoOOm6SVcvgZptEkK6Pj\nJkmiHL6Eoo5JldFxk7QaMOCb2fvM7CEz+6uZrTGzfQve+11tmidxUqGyMjpuklaDXeFfD6wC3gf8\nCbjfzN6Ze2/UQB+S9NDj9yqj4yZpNVjAH+/ud7n7Dne/CjgXuMvMDgeSk/iXikXpmCT9dNwkrQYs\n2prZY8BCd99ZMG8mcAuwn7vvX+3GqGgrIhJNtYq2XwfeWzjD3f8AHAPcWnnzRESkHga8D9/dfzTA\n/OeBf4ytRSIiEgvdltmAovQCVY9RkexQT9sGE6UXqHqMimSLeto2mCi9QNVjVCT9qtrT1szeZWa/\nNLMnctMzzezS4TZS4hGlF6h6jIpkS5gc/neBS4Au6LtT55Q4GyWVi9ILVD1GRbIlTMAf6+6lQyl0\nx9EYGb4ovUDVY1QkW8IE/G25IRUcwMw+BbwYa6ukYlF6garHqEi2DFm0NbODgTXAkcDLwF+Ape5e\nptw3PCraiohEE6VoO+htmWY2Amhz92PNbBwwwt1frUYjRUSktgZN6bh7L8Ggabj7LgV7EZH0CpPD\n/4WZXWRmbzez/fKv2FsmIiJVFaan7X/N/TynYJ4DB1e/OSIiEpchA767v6MWDRERkXgNGfDN7LRy\n8939h9VvjoiIxCVMDn9uwWsBwWMPTxzqQ2bWbGa/M7PHzOyPZvaVYbVU6k4ja4qkW5iUznmF02Y2\nEfjXEOt+A/iAu79mZqMInom73t0fqqypUk8aWVMk/SoZD78TOGSohTzwWm5yVO6VnKE5JZIVK/qD\nfV5nZzBfRNIhTA7/5/QH6hHAdOCnYVZuZk3AJuAfgP/p7r8ts8wyYBnAVI3alVgaWVMk/cLclnlV\nwe/dwGZ33xJm5e7eA7Sa2T7AbWY2w92fKFlmDcHQDbS1tekbQEJNnVp+7Hz9jRZJjzApnQ+5+69z\nrwfcfYuZfT3KRtx9B3AfcHwljZT608iaIukXJuAfV2beCUN9yMwm567sMbMxwLHA09GaJ0mhkTVF\n0m/AlI6ZnQWcDRxsZn8oeGsC8ECIdR8I/CCXxx8B/MTd7xhOY6W+li5VgBdJs8Fy+D8C1gP/Alxc\nMP9Vd//7UCvOPRnrsOE1T0REqmXAgO/uO4GdwKkAZvYWoBkYb2bj3V33Z4iIpEiYh5h/1MyeIXjw\nya+B5wiu/EVEJEXCFG3/GTgc+FNuILVjCJfDFxGRBAkT8LvcfTswwsxGuPu/A60xt0tERKosTMer\nHWY2HtgArDOzlwg6YImISIqEucL/GMH4ORcAdwH/AXw0zkaJiEj1hRktc5eZTQMOcfcfmNlYoCn+\npomISDWFuUvnH4GbgRtysw4Cbo+zUSIiUn1hUjrnAEcBrwC4+zPAW+JslIiIVF+YgP+Gu7+ZnzCz\nkWhcexGR1AkT8H9tZl8GxpjZcQRj4f883maJiEi1hQn4FwNbgceBM4E7gUvjbJSIiFTfYKNlTnX3\n5929F/hu7iUiIik12BV+3504ZnZLDdoiIiIxGizgW8HvB8fdEBERiddgAd8H+F1ERFJosJ62s8zs\nFYIr/TG538lNu7vvHXvrRESkagZ7AIqGTxARaSBhbssUEZEGoIAvIpIRCvgiIhmhgC8ikhEK+CIi\nGaGALyKSEQr4IiIZoYAvIpIRCvgiIhmhgC8ikhEK+CIiGaGALyKSEQr4IiIZoYAvIpIRCvgiIhmh\ngC8ikhEK+CIiGaGALyKSEQr4IiIZoYAvIpIRCvgiIhkRW8A3s7eb2b+b2VNm9kcza49rWyIiMrSR\nMa67G1ju7g+b2QRgk5n9wt2fjHGbIiIygNiu8N39RXd/OPf7q8BTwEFxbU9ERAZXkxy+mbUAhwG/\nLfPeMjPrMLOOrVu31qI56eQ++HRW2iAiFYs94JvZeOAW4AJ3f6X0fXdf4+5t7t42efLkuJuTTqtW\nwYUX9gdY92B61apstUFEhiXWgG9mowiC/Tp3vzXObTUsd9ixA665pj/gXnhhML1jR22uspPQBhEZ\nNvOY/rOamQE/AP7u7heE+UxbW5t3dHTE0p5UKwywee3tsHo1mGWnDSKyBzPb5O5toZaNMeAfDWwA\nHgd6c7O/7O53DvQZBfxBuMOIgi9kvb21D7RJaIOIFIkS8OO8S+d+dzd3n+nurbnXgMFeBpG/ui5U\nmE/PShtEZFjU0zbpClMp7e3BVXV7e3E+PQttEJFhi7PjlVSDGeyzT3G+fPXq4L199qlNSiUJbRCR\nYYsth18J5fAH4V4cWEuns9IGESmSiBy+VFlpYK1HoE1CG0SkYgr4El5Setr29g4+XQtJORYiESjg\nSzhJ6Wm7eDHMmdMf5Ht7g+nFi2vXhqQcC5GIFPBlaEnpadvbCzt3wqOP9gf9OXOC6Z07a3Oln5Rj\nIVIBFW0lnKT0tC0M8nmtrbBpU3GnsDgl5ViIkJCetpVQwE+4pPS07e2Fpqb+6Z6e2gX7vKQcC8k8\n3aWTFlEKf3EVKnt6Bp/Oc4cLSoZEuuCC2qcw8lf4hQpz+rWgXseSUgr49RKl8BdXobKlBQ44oD/I\n9/QE0y0txcu5wxFHwLe+BeefH2z//POD6SOOqG0OP5/OaW0N2tvaWpzTj5t6HUuKKeDXQ5TCX1yF\nyp4e2LULtm3rD/oHHBBM79o18JV+PY0YARMnFufsN20KpidOrE1aZ6Bex+3t6nUsyefuiXnNmTPH\nM6O317293T0I78GrvT2YX6qnx721tXjZ1tZg/nB0d7tPmlS83kmTgvnl2nv++cXLnn9++fbGrXS/\nh3scKlG63/U4DiLuDnR4yBirom09RSn8xVWo7OmBkQVDKnV3F2+n0vaKSE2oaFtPYQuxUQp/UQuV\nYQu8+TROocKcfml7oxRtoxaZk1DAjkI9bSWFFPCrKWwhNkrhL2qhMmyBt6cH9t47yNlPmhRc2U+a\nFEzvvXdx0HcPcuTlirYTJ+4Z7KIWmZNQwI5CPW0lpRTwqyVKITZK4S9KoTJKgXfEiP7UzSmnBNOn\nnBJMNzXtud7u7uD3deuC6XXrgunu7uL1Ri0yJ6GAHYV62kqahU321+KV+qJtlEJsfvnBpguFLVRG\nKfD29rqfe27xsueeW74d3d3u++9fvOz++5cv8EYtMiehgB1F1PMsEiNUtB2Cxziue5QiaBS9vXte\ndQ9UtO3uhlGj+qe7uorbVGl7u7pg9Oj+6TffLN5OpeuF+ArYcZ1rFbAlIVS0HUyc+ddFi8oXQRct\nGt56o+StFy2CCROK502YUL4NUdq7YAGMGVM8b8yYYH6pyy8vv97LL99zWYivgB3XuY7SXpEkCftV\noBav2FM6hV/F81/BS6crVXhPe/5e9tLpShSmMPKpi9LpvK4u9+bm4L3m5vLTlbT3zTfdm5qC95qa\nyk9XehyinJMoxyKucx3nvyGRChAhpVP3IF/4qkkOP87862WXle/IdNllw1tvlLz1ggX9QT7/am4O\n5g+nvVOn9gf5/KupKZhfauHC8utduLD8/q1cWXwO8udo5co9l120qHjf88dm0aI9l43rXEdpr0jM\nogT87Obw48q/xpnDD5u3TmsOP2yuPUo9I65zHaW9IjFSDn8w7sHtj4Xa2wfvIDXYdKHeXmgrOe5t\nbQPfLhh23VHy1r29MHdu8by5cwdeNmx7e3th3rziefPmlV/WHZYvL563fPngxy7K83LDLuse3wif\ner6vpFHYrwK1eNUkh3/QQcFX+/POC6bPOy+YPuigPb/qR/nqHiW/HGXdUdYbddkDDii/7AEHVL7e\nuHPcYY9bb6/7/PnBdvNj/uTHApo/X7l2aRhESOlk6wrfvb8H6YYNwfSGDcF0T0/xlZ97tA42UTpI\nRVl3lPVGWdasP8WyYEEwnb/jpqmp8s5fcY4mGfWciEixsH8ZavGqSdE2asekqEW/sB2koq47ygiR\nUdqQ/4aTf+W/+Qy3DXGNJhnluCVphE+RmKCi7RCidtyJq8Ab57rT1IaoorQ5jfsnEoGKtoOJUgDN\npwwKVauDTZzrjtKGJDy2MIooxy0JxzjfjsGmRWol7FeBWrxiT+kkpfiYhM47aSxqRjluSTjG7rpn\nX2JHhJTOADdnN6iBio9z5oQvPsLwi49xrruRRTluSTjGXlBkhmD7hcNiu+7dl9rKbg4/SseduDrY\nxLnusNu/4IJgXPu888+Hq69OdiCKctyScIzzQT6v8I+QyDBFyeFnM+BLPxU146djLDHKZtFWhbHo\nklLUbGQ6xpIgjRHw9ci56ApTDUM9ZlEqo2MsCZP+oq0KY5VJQlGz0ekYS8I0Rg5fhbHK1buomQU6\nxhKjbBZtk1IY039uEamhRBRtzex/mdlLZvZEXNvok5TCmGoJIpJgcRZt/zdwfIzrDySlMFZYS9BI\njiKSQLEVbd39N2bWEtf6+ySlMFa43Wuu6a8nqJYgIgkRaw4/F/DvcPcZgyyzDFgGMHXq1DmbN2+u\nbGNJyZ0npZYgIpmQiBx+WO6+xt3b3L1t8uTJla8oCY+cS0otQUSkjLoH/IaRlFqCiMgA0t/xKimS\nUksQERlAbDl8M/s3YDEwCfgbsNLdvzfYZxpi8LSk1BJEJBOi5PDjvEvn1LjWnWhJqCWIiJShHL6I\nSEYo4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGREosbDN7OtQIWD6cRuErCt3o2IkfYv3Rp5/xp5\n32D4+zfN3UONS5OogJ9kZtYRtnNDGmn/0q2R96+R9w1qu39K6YiIZIQCvohIRijgh7em3g2ImfYv\n3Rp5/xp536CG+6ccvohIRujfaNa6AAAFaklEQVQKX0QkIxTwRUQyQgG/DDNrMrNHzOyOMu+dbmZb\nzezR3OuMerSxUmb2nJk9nmv7Hg8fsMC3zOxZM/uDmc2uRzsrFWL/FpvZzoLzd3k92lkJM9vHzG42\ns6fN7CkzO6Lk/bSfu6H2L83n7t0F7X7UzF4xswtKlon9/OmJV+W1A08Bew/w/k3ufm4N21Nt73f3\ngTp6nAAcknvNB67P/UyTwfYPYIO7f6Rmramea4C73P1TZjYaGFvyftrP3VD7Byk9d+7+f4FWCC4o\ngf8H3FayWOznT1f4JcxsCvBh4MZ6t6VOPgb80AMPAfuY2YH1blTWmdnewELgewDu/qa77yhZLLXn\nLuT+NYpjgP9w99JRBWI/fwr4e7oa+G9A7yDLnJT7ynWzmb29Ru2qFgfuMbNNZraszPsHAX8tmN6S\nm5cWQ+0fwBFm9piZrTezQ2vZuGE4GNgKfD+XbrzRzMaVLJPmcxdm/yCd567UKcC/lZkf+/lTwC9g\nZh8BXnL3TYMs9nOgxd1nAvcCP6hJ46rnKHefTfD18RwzW1jyfrlnMqbp3t2h9u9hgrFHZgHfBm6v\ndQMrNBKYDVzv7ocBu4CLS5ZJ87kLs39pPXd9cqmqE4Gflnu7zLyqnj8F/GJHASea2XPAj4EPmNna\nwgXcfbu7v5Gb/C4wp7ZNHB53fyH38yWCHOK8kkW2AIXfWqYAL9SmdcM31P65+yvu/lru9zuBUWY2\nqeYNjW4LsMXdf5ubvpkgQJYuk9ZzN+T+pfjcFToBeNjd/1bmvdjPnwJ+AXe/xN2nuHsLwdeuX7n7\n5wqXKcmpnUhQ3E0FMxtnZhPyvwNLgCdKFvsZcFrujoHDgZ3u/mKNm1qRMPtnZgeYBU+WN7N5BP8H\ntte6rVG5+38CfzWzd+dmHQM8WbJYas9dmP1L67krcSrl0zlQg/Onu3RCMLOvAh3u/jPgfDM7EegG\n/g6cXs+2RfRW4Lbc/5mRwI/c/S4z+ycAd/8OcCfwIeBZoBP4fJ3aWokw+/cp4Cwz6wZeB07x9HQ3\nPw9Yl0sL/Bn4fAOdOxh6/9J87jCzscBxwJkF82p6/jS0gohIRiilIyKSEQr4IiIZoYAvIpIRCvgi\nIhmhgC8ikhEK+NIwzKynZETClgrWsY+ZnV391vWt/z1mttHM3jCzi+Lajkg5ui1TGoaZvebu44e5\njhbgDnefEfFzTe7eE2K5twDTgI8DL7v7VZW0U6QSusKXhmbBsw2+YWa/zw14d2Zu/ngz+6WZPWzB\n+Pkfy33kvwPvzH1D+IYFY7DfUbC+a83s9Nzvz5nZ5WZ2P/BpM3unmd2VG7htg5m9p7Q97v6Su/8e\n6Ip950VKqKetNJIxZvZo7ve/uPsngC8QdFGfa2Z7AQ+Y2T0EoxJ+wt1fyY3H8pCZ/YxgwK4Z7p4f\nu3zxENvc7e5H55b9JfBP7v6Mmc0HrgM+UO2dFKmUAr40ktfzgbrAEmCmmX0qNz2R4AETW4Arc6Np\n9hIMQ/vWCrZ5EwTfGIAjgZ/mhnYA2KuC9YnERgFfGp0B57n73UUzg7TMZGCOu3flRkhtLvP5bopT\nn6XL7Mr9HAHsKPMHRyQxlMOXRnc3wYBbowDM7F25kTQnEjz7oMvM3k9QSAV4FZhQ8PnNwHQz28vM\nJhKM4rgHd38F+IuZfTq3HTOzWfHskkhldIUvje5GoAV4ODe07laCO2TWAT+34EHnjwJPQ/C8AzN7\nwMyeANa7+xfN7CfAH4BngEcG2dZS4HozuxQYRfBMhccKFzCzA4AOgucl91rwIOvpuT8YIrHSbZki\nIhmhlI6ISEYo4IuIZIQCvohIRijgi4hkhAK+iEhGKOCLiGSEAr6ISEb8f7vahkhdNuLQAAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa219b70>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Original Data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. PLA算法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征归一化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先分别对两个特征进行归一化处理，即："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$X=\\frac{X-\\mu}{\\sigma}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，$\\mu$是特征均值，$\\sigma$是特征标准差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEWCAYAAACaBstRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X2cVnWd//HXhwEEBFGBFgNhrMVN\nJJhgALVUdr2vVjPTMLayjcUfeYP+sE2XFH62uPWoX95k3tB9K1uamplJmLWupNgyGCneJGSiE24C\nCSojwsx89o9zrplrLq4Zzpm5znXOdV3v5+NxPWbOzZzzub4D12fO93O+32PujoiISBT90g5AREQq\nh5KGiIhEpqQhIiKRKWmIiEhkShoiIhKZkoaIiESmpCESkZktMbPbwu/HmdkbZlZX4nMca2a/L+Ux\nezjXC2Z2YjnOJdVDSUMyI/wQ+7OZ7Z+3bq6ZPZRiWEW5+4vuPtTd2/pyHDNzM/vrvOOucve/6XuE\npVUYp9QuJQ3Jmv7Agr4exAL69y1SYvpPJVnzZeAyMzuw2EYzO8bM1pjZjvDrMXnbHjKzpWb2CNAC\nvCNc969m9mjYnfRTMxthZsvN7LXwGPV5x7jezF4Kt601s2O7iaM+/Ou7v5kdHR4799plZi+E+80w\ns9Vmtt3MXjazG81sYLjt4fBwvwt/7qNmNsvMmvPOc0T4Hrab2VNmdnretu+a2dfN7Gdm9rqZ/cbM\n3tldw5rZx81sk5ltM7NFBdvixnmQmd1nZlvM7NXw+7HdnVuqh5KGZE0T8BBwWeEGMzsY+BlwAzAC\n+CrwMzMbkbfbx4F5wDBgU7hudrh+DPBOYDXwHeBg4Blgcd7PrwEawm3/AfzIzAb1FLC7rw67qoYC\nBwGPAT8IN7cBlwIjgaOBE4DPhD93XLjPlPDnby94vwOAnwIPAG8DLgKWm1l+99W5wP8Lz7sRWFos\nRjObCNwctsPbCdov/0M+bpz9CNpwPDAOeBO4sad2kuqgpCFZdBVwkZmNKlj/AWCDu/+7u7e6+w+A\nZ4G/z9vnu+7+VLh9T7juO+7+B3ffAawA/uDuD7p7K/Aj4D25H3b329x9W/jz/x/YD4hTY7gB2Aks\nCo+31t0fC4/3AnArcHzEYx0FDAW+6O673f1XwH0EiSLnbnf/7/C9LCdIeMV8BLjP3R9297eAK4H2\n3Ma4cYZtdJe7t7j76wTJKur7kgrWP+0ARAq5+3ozuw+4nOBKIOftdF495GwiuILIeanIIf+c9/2b\nRZaH5hbMbCEwNzyXAwcQ/PW9T2Z2PjALOMrd28N1hxNcETUCQwj+z62NcrwwhpdyxwoVvt//yfu+\nJf+9FDtWbsHdd5rZtrzYY8VpZkOAa4FTCa5yAIaZWV1fbw6QbNOVhmTVYuCf6PoBuZmgOyTfOOBP\necu9nrY5rF98DjgHOMjdDwR2ABbxZ78AnBFe0eTcTHA1NMHdDwD+JcrxQpuBQwsK+oXvN6qXgUPz\n4h1C0EXV2zgXElyBzQz3z3VhRX1vUqGUNCST3H0jcDtwcd7q+4HDzexjYQH6o8BEgi6bUhgGtAJb\ngP5mdhXBlUaPzOzQMNZPuPtzRY75GvCGmb0LmF+w/c/AO7o59G8Iurr+2cwGmNksgq64H0Z7O13c\nCXzQzN4XFrivpuv//7hxDiO4Stse1poWIzVBSUOy7GqgY8yGu28DPkjwV+424J+BD7r71hKdbyVB\nzeM5gm6gXRTv7ip0AjAauDPvDqqnwm2XAR8DXge+QZBc8i0BvhfetXRO/gZ33w2cDpwGbAVuIkhM\nz8Z9Y+7+FHABQXH/ZeBVoDlvl7hxXgcMDuN6DPh53JikMpkewiQiIlHpSkNERCJT0hARkciUNERE\nJDIlDRERiazqBveNHDnS6+vr0w5DRKSirF27dqu7F87CsJeqSxr19fU0NTWlHYaISEUxs8LZFopS\n95SIiESmpCEiIpEpaYiISGRVV9MoZs+ePTQ3N7Nr1660Q6lIgwYNYuzYsQwYMCDtUEQkZTWRNJqb\nmxk2bBj19fWYaRLOONydbdu20dzczGGHHZZ2OCKSsprontq1axcjRoxQwugFM2PEiBG6ShMRoEaS\nBqCE0Qdqu+q3fDnU10O/fsHX5cvTjkiyqia6p0Ske8uXw7x50NISLG/aFCwDzJmTXlySTTVzpZG2\nuro6GhoamDRpEmeffTYtuf+hMcydO5enn34agGuuuabLtmOOOaYkcUrtWbSoM2HktLQE60UKKWmU\nyeDBg1m3bh3r169n4MCB3HLLLbGP8c1vfpOJEycCeyeNRx99tCRxSu158cV466W2pZo0zOzbZvaK\nma3vZvssM9thZuvC11XliCvp/t1jjz2WjRs3AvDVr36VSZMmMWnSJK677joAdu7cyQc+8AGmTJnC\npEmTuP324CFqs2bNoqmpicsvv5w333yThoYG5oT9B0OHDgXgox/9KPfff3/Huc477zzuuusu2tra\n+OxnP8v06dOZPHkyt956a2nflFSscePirZca5+6pvQgeRj8VWN/N9lnAfXGOOW3aNC/09NNP77Wu\nO7fd5j5kiDt0voYMCdb3xf777+/u7nv27PHTTz/db7rpJm9qavJJkyb5G2+84a+//rpPnDjRH3/8\ncb/zzjt97ty5HT+7fft2d3c//vjjfc2aNV2OV3j8u+++2z/xiU+4u/tbb73lY8eO9ZaWFr/11lv9\nC1/4gru779q1y6dNm+bPP/985PjjtKFUlqT+zdeC225zHz/e3Sz4WsltBjR5hM/YVK803P1h4C9p\nxlAoqf7d3JVBY2Mj48aN49Of/jS//vWvOfPMM9l///0ZOnQoH/7wh1m1ahXvfve7efDBB/nc5z7H\nqlWrGD58eOTznHbaafzqV7/irbfeYsWKFRx33HEMHjyYBx54gO9///s0NDQwc+ZMtm3bxoYNG/r2\npqQqzJkDy5bB+PFgFnxdtkxF8H3J3UCwaVOQanM3EFT7nWeVcPfU0Wb2O2AzcJm7P1W4g5nNA+YB\njOvjNXVS/bu5mkY+7+b57Icffjhr167l/vvv54orruDkk0/mqqui9cwNGjSIWbNmsXLlSm6//XbO\nPffcjnN97Wtf45RTTunbG5GqNGeOkkRcPf2BWc1tmfVC+OPAeHefAnwNuKfYTu6+zN0b3b1x1Kh9\nTgffo3L27x533HHcc889tLS0sHPnTn784x9z7LHHsnnzZoYMGcI//MM/cNlll/H444/v9bMDBgxg\nz549RY87e/ZsvvOd77Bq1aqOJHHKKadw8803d/zMc889x86dO0v/pkRqRK3eQJDppOHur7n7G+H3\n9wMDzGxkkudcuhSGDOm6bsiQYH2pTZ06lfPOO48ZM2Ywc+ZM5s6dy3ve8x6efPJJZsyYQUNDA0uX\nLuXzn//8Xj87b948Jk+e3FEIz3fyySfz8MMPc+KJJzJw4EAguF134sSJTJ06lUmTJnH++efT2tpa\n+jclUiNq9gaCKIWPJF9APd0XwkcDFn4/A3gxt9zdq6+FcPfqKm6VigrhkrRK+3+X5A0EabQFEQvh\nqdY0zOwHBHdIjTSzZmAxMADA3W8BPgLMN7NW4E1gdvjmEqX+XZHyqsRR6bm4Fi0KuqTGjQt6JPoa\nb9bbwsrwGVxWjY2NXvi412eeeYYjjjgipYiqg9pQklRfH3w4Fho/Hl54odzRpCuttjCzte7euK/9\nMl3TEJHaUKtF5WKy3hZKGiJVqpJmrq2FonLU30fW20JJQ6QKVdrAs3LetZiGOL+PrLeFkoZIFaq0\nmWurfVR6nN9H1ttCSaNMzIyFCxd2LH/lK19hyZIlJT+PpkwXyH6/eDFz5gSF3vb24GtWPiRLIe7v\nI8ttoaRRTOEdZSW4w2y//fbj7rvvZuvWrX0+Vk80ZbpA9vvFa001/T6UNAotWQKXXtqZKNyD5T5e\nFfTv35958+Zx7bXX7rVty5YtnHXWWUyfPp3p06fzyCOPdKw/6aSTmDp1Kueffz7jx4/vSDof+tCH\nmDZtGkceeSTLli0D0JTp0iHJfvHPfAb69w+6Tvr3D5ZLoZIK9zlRY07y91H2dosyArCSXn0aEd7e\n7r5gQTC0c8GC4su9tP/++/uOHTt8/Pjxvn37dv/yl7/sixcvdnf3c88911etWuXu7ps2bfJ3vetd\n7u5+wQUX+DXXXOPu7itWrHDAt2zZ4u7u27Ztc3f3lpYWP/LII33r1q0d5yk8r3vfp0zXiPDKk8So\n4vnzu46Azr3mz+97rJU2PXvcmJP4fZSy3Yg4Ijz1D/lSv/o8jUh+osi9+pgw3Ds/vK+88kq/+uqr\nuySNUaNG+ZQpUzpeb3/72/21117zKVOmdPkAP+iggzqSxuLFi33y5Mk+efJkP+CAA3z16tVdzlN4\n3jfffNPHjh3ru3bt8nvuucc/9rGPubv7WWed5RMmTOg4d319va9cuXKv+JU0xN29rq540qir69tx\nx48vftzx40sRdTKyEHMpY4iaNCphavTyMoNrr4Xrr+9cd+21wfoSuOSSS5g6dSqf+tSnOta1t7ez\nevVqBg8e3GXf4Pe4t4ceeogHH3yQ1atXM2TIEGbNmsWuXbt6PK+mTJdSaGuLtz6qSizcZyHmNGJQ\nTaOQhzWMfPk1jj46+OCDOeecc/jWt77Vse7kk0/mxhtv7FjOPXfjfe97H3fccQcADzzwAK+++ioA\nO3bs4KCDDmLIkCE8++yzPPbYYx0/qynTJUl1dfHWR1WJheIsxJxGDEoa+XIJ4/rrYcGC4H63BQuC\n5RImjoULF3a5i+qGG26gqamJyZMnM3HiRG655RYAFi9ezAMPPMDUqVNZsWIFhxxyCMOGDePUU0+l\ntbWVyZMnc+WVV3LUUUd1HEtTpkuSchPnRV0fVdID2uIUi7NQ3I4qlRii9GFV0qvPNY3Fi7vWMHI1\njrD+UE67du3yPXv2uLv7o48+6lOmTCl7DDmqaYh7UGDt379r/3n//tmeDjxOsTgLxe24ShUDEWsa\nmuW2GPeuNYzC5TLZsGED55xzDu3t7QwcOJCbbrqJ6dOnlz0O0Cy3EqjE2WjjxFyJ769Uos5yq0J4\nMYUJIoWEATBhwgR++9vfpnJukWKyUPyNK07Mlfj+yq1mahrVdkVVTmo7yclC8TeuODFX4vsrt5pI\nGoMGDWLbtm368OsFd2fbtm0MGjQo7VBKLqmRtEkUXZMWdZR33MJrFto4TsxZKG5nXU10T40dO5bm\n5ma2bNmSdigVadCgQYwdOzbtMEoqqUdqxjluVh7r+ZnPwM03dy63tXUu33RT133jPOI0C20cN+ak\nHuFaTWqiEC5SKKmCZyUWXfv3Lz44r64O+nL3dRbaWKKLWghX0pCa1K9f8WE3ZsHwnHIcN6kY4urp\nPo++fDxkoY0lOj0jXKQHSRU8K7HoWmmjvLPSbrVKSUNqUlIF3UosumZllHdWRmJX4o0MZRVlBGAl\nvYqNCBcpJupI2iRHCWdhRLF7MLV5bgbburq+T3Wek4U2jhtvUqPHsw6NCBcpDRVek5eVNq7EGxlK\nRYVwkRJR4TV5WWnjSryRoVQqohBuZt82s1fMbH03283MbjCzjWb2hJlNLXeMIiq89l7UPv+4bZxU\nLaESb2Qot7QL4d8FTu1h+2nAhPA1D7i5h31FEpGVgnWlyQ3C27Qp+Is8Nwiv2Ad8nDaOc9y4KvFG\nhrKLUvhI8gXUA+u72XYrcG7e8u+BQ3o6ngrhkoSsFKwrSdxHkUZt46Qfs1qJNzKUApVSCDezeuA+\nd59UZNt9wBfd/dfh8i+Bz7l7U8F+8wiuRBg3bty0TcWqUyJSVhrcV1kqoqYRQbGxqnv9c3H3Ze7e\n6O6No0aNKkNYIrIvGtxXnbKeNJqBQ/OWxwKbU4pFJJI4Rdqos8smHUcSkurzr9laQlZE6cNK8kXP\nNY0PACsIrjiOAv57X8dTTUPSFGfA1/z5xfvmSzGwLisDz5IchFcttYSsoBJqGmb2A2AWMBL4M7AY\nGADg7reYmQE3Etxh1QJ8ygvqGYU0TkPSFGfAV1Kzy8aNQwQ0uC/tMKRGxSnSJjW7bNw4RKB6CuEi\nFSVOkTap2WXjxiESh5KGSAktXQoDB3ZdN3Bg8SJt3Nllk3rEqUgcShoiJVbYLdRdV9N73xvUNfL1\n7x+sLxR3FPScObBsWVDDMAu+Llumx5ZK36mmIVJCSc2SqsK2JE01DZEUvPhi9PVJ7SuSJCUNkRJK\napZUFbYlK5Q0RCKIOnI7qVlSly6FAQO6rhswQI84lRREGQFYSS+NCJdSiztyO4lZUm+7zX3gwK7n\nHziwNCOha/kRp9KJShgRngQVwqXUkhy5HVWShXAV5AVUCBcpmWIJo6f1SUiyEK6CvMShpCGZloX+\n87gjt5OIOclCuAryEoeShmRWko/1jCPOyO2kYn7/++Otj0OPOJVYohQ+KumlQnj1SPqxnnHMn+9e\nVxecv66u+yJ4UjHrEaeSNFQIl0pXiTO16hGnUqlUCJeKV4n953rEqVQ7JQ3JrCz1n0ctbusRp1L1\novRhVdJLNY3qkoX+87gD2vSIU6lEqKYhUhoa0Ca1QDUNkRLRgDaRTkoaIvugIrRIJyUNkX1IeoZZ\nkUqipCESgVnPyyK1QklDZB8WLYLdu7uu2707WC9Sa5Q0RPZBhXCRTkoaIvugQrhIp1SThpmdama/\nN7ONZnZ5ke3nmdkWM1sXvuamEafUtrijsbMwnbtIUvqndWIzqwO+DpwENANrzOxed3+6YNfb3f3C\nsgcoEpozJ/i6aFHQJTVuXJAwcuvz5aZGb2kJlnNTo+cfR6SSpXmlMQPY6O7Pu/tu4IfAGSnGI9Kt\nOXOC0d/t7cHX7hLAokWdCSOnpUVFc6keaSaNMcBLecvN4bpCZ5nZE2Z2p5kdWuxAZjbPzJrMrGnL\nli1JxCoSiYrmUu3STBrF7nQvnAjrp0C9u08GHgS+V+xA7r7M3RvdvXHUqFElDlOiUD9+IG7RXO0m\nlSbNpNEM5F85jAU25+/g7tvc/a1w8RvAtDLFJjFk5bGsWRCnaK52k0qUZtJYA0wws8PMbCAwG7g3\nfwczOyRv8XTgmTLGJxGpH7/TnDmwbFkwA65Z8HXZsuI1ELWbVKJUp0Y3s/cD1wF1wLfdfamZXU0w\nr/u9ZvZvBMmiFfgLMN/dn+3pmJoavfz0KNLeUbtJlvR5anQze7eZPWZmL5nZMjM7KG/bf5ciSHe/\n390Pd/d3uvvScN1V7n5v+P0V7n6ku09x97/dV8KQdGjwW++o3aQS9dQ9dTOwBHg38BzwazN7Z7ht\nQHc/JLVHjyLtHbWbVKKeksZQd/+5u293968AFwI/N7Oj2PsuJ6lhcfrxpZPaTSpRtzUNM/sdcJy7\n78hbNxm4CzjY3UeUJ8R4VNMQEYmvFI97/RJwRP4Kd38COAG4u2/hiYhIJep27il3/49u1r8I/FNi\nEYmISGZpanTpVpzRyhrZLFIbUpvlVrItzmytmtlVpHakOrgvCSqEl0Z9ffDhX2j8+GCW197uKyLZ\nVIpCeO5Ah5vZL81sfbg82cw+X4ogJbvizNaqmV1FakeUmsY3gCuAPdBxB9XsJIOS9MUZrayRzSK1\nI0rSGOLuhdOGtCYRjGRHnNHKGtksUjuiJI2t4fQhDmBmHwFeTjQqSV2c0coa2SxSO/ZZCDezdwDL\ngGOAV4E/AnPcvUjpM30qhIuIxBe1EN7jLbdm1g9odPcTzWx/oJ+7v16qIEVEpLL02D3l7u0EExXi\n7juVMEREaluUmsYvzOwyMzvUzA7OvRKPTEREMifKiPB/DL9ekLfOgXeUPhwREcmyfSYNdz+sHIGI\niEj27TNpmNkniq139++XPhwREcmyKDWN6XmvYwkeAXt6gjFJldOMuCKVK0r31EX5y2Y2HPj3xCKS\nqqYZcUUqW2+ep9ECTCh1IFIbFi3qTBg5LS3BehHJvig1jZ8STiFCkGQmAj9KMiipXpoRV6SyRbnl\n9it537cCm9y9OaF4pMqNG1f82RuaEVekMkTpnnq/u/9X+HrE3ZvN7EuJRyZVSTPiilS2KEnjpCLr\nTit1IFIbNCOuSGXrNmmY2XwzexL4GzN7Iu/1R+CJUpzczE41s9+b2UYzu7zI9v3M7PZw+2/MrL4U\n55V0zZkTPAa2vT34qoQhUjl6qmn8B7AC+Dcg/wP9dXf/S19PbGZ1wNcJrmSagTVmdq+7P52326eB\nV939r81sNvAl4KN9PbeIiPROt1ca7r7D3V9w93PDZ2e8SXAX1VAzK0XZcgaw0d2fd/fdwA+BMwr2\nOQP4Xvj9ncAJZmYlOLeIiPTCPmsaZvb3ZraB4OFL/wW8QHAF0ldjgJfylpvDdUX3cfdWYAcwokiM\n88ysycyatmzZUoLQRESkmCiF8H8FjgKeCycvPAF4pATnLnbFUPgYwSj74O7L3L3R3RtHjRpVgtBE\nRKSYKEljj7tvA/qZWT93/0+goQTnbgYOzVseC2zubh8z6w8MB/pcTxERkd6JMrhvu5kNBVYBy83s\nFYJBfn21BphgZocBfwJmAx8r2Ode4JPAauAjwK98Xw81FxGRxES50jiDYL6pS4CfA38A/r6vJw5r\nFBcCK4FngDvc/Skzu9rMcrPofgsYYWYbgf9L17u4RESkzKLMcrvTzMYDE9z9e2Y2BKgrxcnd/X7g\n/oJ1V+V9vws4uxTnEhGRvoty99Q/Edzuemu4agxwT5JBiYhINkXpnroAeC/wGoC7bwDelmRQIiKS\nTVGSxlvh4Dug4y4mFaNFRGpQlKTxX2b2L8BgMzuJ4FkaP002LBERyaIoSeNyYAvwJHA+QeH680kG\nJSIi2dTt3VNmNs7dX3T3duAb4UtERGpYT1caHXdImdldZYhFREQyrqekkT/v0zuSDkRERLKvp6Th\n3XwvIiI1qqcR4VPM7DWCK47B4feEy+7uByQenYiIZEq3ScPdSzJViIiIVI8ot9yKiIgAShoiIhKD\nkoaIiESmpCEiIpEpaYiISGRKGiIiEpmShoiIRKakISIikSlpiIhIZEoaIiISmZKGiIhEpqQhIiKR\nKWmIiEhkShoiIhJZKknDzA42s1+Y2Ybw60Hd7NdmZuvC173ljlNERLpK60rjcuCX7j4B+GW4XMyb\n7t4Qvk4vX3giIlJMWknjDOB74fffAz6UUhwiIhJDWknjr9z9ZYDw69u62W+QmTWZ2WNm1m1iMbN5\n4X5NW7ZsSSJeERGh52eE94mZPQiMLrJpUYzDjHP3zWb2DuBXZvaku/+hcCd3XwYsA2hsbPReBSwi\nIvuUWNJw9xO722ZmfzazQ9z9ZTM7BHilm2NsDr8+b2YPAe8B9koaIiJSHml1T90LfDL8/pPATwp3\nMLODzGy/8PuRwHuBp8sWoYiI7CWtpPFF4CQz2wCcFC5jZo1m9s1wnyOAJjP7HfCfwBfdXUlDRCRF\niXVP9cTdtwEnFFnfBMwNv38UeHeZQxMRkR5oRLiIiESmpCEiIpEpadQa956XayUGEekVJY1asmQJ\nXHpp54e0e7C8ZEltxSAivaakUSvcYft2uP76zg/tSy8NlrdvL89f+1mIQUT6xLzK/qM2NjZ6U1NT\n2mFkU/6HdM6CBXDttWBWOzGIyF7MbK27N+5zPyWNGuMO/fIuMNvby/9hnYUYRKSLqElD3VO1JPdX\nfr78+kKtxCAivaakUSvyu4UWLAj+ul+woGt9oRZiEJE+SWVEuKTADA48sGv94Nprg20HHlie7qEs\nxCAifaKaRq1x7/rhXLhcKzGISBeqaUhxhR/Otfxh3d7e83I5aKCjVBglDSmvrAzumzULpk3rTBTt\n7cHyrFnliyErbSESg5KGlE9WBve1t8OOHbBuXWfimDYtWN6xozxXHFlpC5GYVNOQ8srK4L78RJHT\n0ABr13YdQ5KkrLSFCBrcl3YY0pOsDO5rb4e6us7ltrbyJYycrLSF1DwVwmtJnGJqUsXftrael3Pc\n4ZJLuq675JLyd8fkrjTy5dc4ykEDHaUCKWlUujjF1KSKv/X1MHp0Z6JoawuW6+u77ucORx8NN9wA\nF18cnP/ii4Plo48ub00j1zXV0BDE29DQtcaRNA10lAqlpFHJ4hRTkyr+trXBzp2wdWtn4hg9Olje\nubP7K4409esHw4d3rWGsXRssDx9eni6q7gY6LliggY6SaappVLo4xdSkir/5iSJn5Ej4n//pWjPI\nxXvJJcHVRc7FF8N115X/g7K9fe96Qho1DQ10lAxQIbyWxCmmJlX8bWuD/nmz0rS27p0wehOviJSF\nCuGVLE5hO04xNW7xN2rRPHelkS+/xlEYb5xCeNzCfRZuCohDI8KlwihpZE2cwnacYmrc4m/Uonlb\nGxxwQNA1NXJkcIUxcmSwfMABXROHe1AzKFYIHz587w/MuIX7LNwUEIdGhEsFUtLIkrijhOMUU+MU\nf+MUzfv16+yGmj07WJ49O1iuq9v7uK2twffLlwfLy5cHy62tXY8bt3CfhZsC4tCIcKlU7l5Vr2nT\npnlFa293X7DAPfjYCF4LFgTre/qZnpbztbX1vJy/vqGhaxwNDcX3b293v/DCrvteeGHxOFpb3UeM\n6LrviBHB+r7EkIsjatvFPXYSevO7FkkI0OQRPmNTKYSb2dnAEuAIYIa7F61cm9mpwPVAHfBNd//i\nvo5d1kJ4Une+xCkqxxXnjqHWVhgwoHN5z56uceWLE/OePTBwYOfy7t1dz9Pb40JyNwUk9bvWTQGS\nEVkvhK8HPgw83N0OZlYHfB04DZgInGtmE8sTXgRJ9Ucff3zxovLxx/ftuBCvH//442HYsK7rhg0r\nHkecmI89FgYP7rpu8OBgfaGrrip+3Kuu2ntfSO6mgKR+13HiFcmKKJcjSb2Ah4DGbrYdDazMW74C\nuGJfxyxL91R+t0KuO6FwuTdaW91HjgyOM3Jk8eXeyu+OyXXDFC7n7NnjPmhQsG3QoOLLvYl59273\nurpgW11d8eXetkWc30mctkjqd53UcUV6iYjdU1l+3OsY4KW85WZgZrEdzWweMA9g3LhxyUeW/5jS\n66/vHFjX1xlK6+pg/ny4+ebg7qNct8zIkcH6vnRR5QrfuYJv7ljFBvf17w/Tp8OaNbBrV2fX0aBB\nwfr87qI4MQ8YAGPGwJ/+FHTXFHW/AAAIGklEQVQF5bqo6uqC9fldVHV1MHEiPP303sedOHHvtojz\nKNnubgqYNm3vmwKS+l3r0bdSoRKraZjZg8DoIpsWuftPwn0eAi7zIjWNsO5xirvPDZc/TlD/uKin\n85a9ppFEf3TSNY2o/fiVWtOIWnuIU99J6nedVK1EJKbUaxrufqK7Tyry+knEQzQDh+YtjwU2lz7S\nXnIP/krMt2BB9/3Rheu726+9HRoLfm+NjT3fBhrn2FH78dvbg6uKfNOnd79v1Jjb22HGjK7rZswo\nvq87LFzYdd3ChT33+cd5nG3UfT3mgMQ49PhdqTRR+rCSetFzTaM/8DxwGDAQ+B1w5L6OWbaaxpgx\nQf/zRRcFyxddFCyPGbN3f/TixV37qXP914sXd90vTl97kseOu+/o0cX3HT2698dNus8/aru1t7vP\nnBmc9+KLg+WLLw6WZ85U7UGqBhFrGqncPWVmZ5pZM0Gx+2dmtjJc/3Yzux/A3VuBC4GVwDPAHe7+\nVBrx7sW9c6TzqlXB8qpVwXJbW9e/QD3GIK64s68mdew4+5p1dhcde2ywnLsTqq6u9wMMk5wFNk67\niUhXUTJLJb3KNrgv7uC3OIO4og7AS/rYUffNv9LKvXJXYH2NIc7AxTjitFv+1UXulbvqEKkSZHlw\nX5LKWgiPOzgsqUFcSR67kmKIK07Mlfj+RGJIvRBe9eIUlXPdH/lKNYgryWPHiSELj3CNI067ZaGN\nc3H0tCxSDlEuRyrpVZbuqawUdLMwQKwSC8Vx2i0LbewevXAv0ktUweC+7Io7OCypQVwaINY7cdot\nC23seYV7CM6fPyW+a2yHlI9qGn0Rd3BYUoO4kjx21PNn5RGuccRptyy0cdTH+or0gh73KuWlQnHy\n1MaSIBXCe0vFxviyUiiuZmpjyQgljXx6/GZ8+d0m+3rkrPSO2lgyRIXwHBUbeycLheJqpzaWDFFN\nI5+Kjb2XdqG4FqiNJUEqhPdWFoqN+nAQkTJTIbw3slBsVF1FRDJMSSMnC8XG/LqKZl8VkQxSITwn\nC8XGpB4tKiJSIqppFMpCPSELdRURqSmqafRW2o/fzEJdRUSkG0oaWZKFuoqISA9U08iSLNRVRER6\noJpGFmWhriIiNUU1jUqWdl1FRKQbShoiIhKZkoaIiESmpCEiIpEpaYiISGRKGiIiEpmShoiIRFZ1\n4zTMbAuwqcimkcDWModTKdQ2PVP7dE9t071Ka5vx7j5qXztVXdLojpk1RRm4UovUNj1T+3RPbdO9\nam0bdU+JiEhkShoiIhJZLSWNZWkHkGFqm56pfbqntuleVbZNzdQ0RESk72rpSkNERPpISUNERCKr\nqaRhZl82s2fN7Akz+7GZHZh2TFlhZmeb2VNm1m5mVXebYG+Y2alm9nsz22hml6cdT5aY2bfN7BUz\nW592LFljZoea2X+a2TPh/6kFacdUSjWVNIBfAJPcfTLwHHBFyvFkyXrgw8DDaQeSBWZWB3wdOA2Y\nCJxrZhPTjSpTvgucmnYQGdUKLHT3I4CjgAuq6d9OTSUNd3/A3VvDxceAsWnGkyXu/oy7/z7tODJk\nBrDR3Z93993AD4EzUo4pM9z9YeAvaceRRe7+srs/Hn7/OvAMMCbdqEqnppJGgX8EVqQdhGTWGOCl\nvOVmqug/vpSHmdUD7wF+k24kpdM/7QBKzcweBEYX2bTI3X8S7rOI4BJyeTljS1uUtpEOxZ6xq/vT\nJTIzGwrcBVzi7q+lHU+pVF3ScPcTe9puZp8EPgic4DU2SGVfbSNdNAOH5i2PBTanFItUGDMbQJAw\nlrv73WnHU0o11T1lZqcCnwNOd/eWtOORTFsDTDCzw8xsIDAbuDflmKQCmJkB3wKecfevph1PqdVU\n0gBuBIYBvzCzdWZ2S9oBZYWZnWlmzcDRwM/MbGXaMaUpvGHiQmAlQSHzDnd/Kt2ossPMfgCsBv7G\nzJrN7NNpx5Qh7wU+Dvxd+Dmzzszen3ZQpaJpREREJLJau9IQEZE+UNIQEZHIlDRERCQyJQ0REYlM\nSUNERCJT0hApYGZtebdKrgungoh7jAPN7DOlj67j+O8ys9Vm9paZXZbUeUQK6ZZbkQJm9oa7D+3j\nMeqB+9x9Usyfq3P3tgj7vQ0YD3wIeNXdv9KbOEXi0pWGSARmVhc+j2VN+DyW88P1Q83sl2b2uJk9\naWa5mXC/CLwzvFL5spnNMrP78o53o5mdF37/gpldZWa/Bs42s3ea2c/NbK2ZrTKzdxXG4+6vuPsa\nYE/ib14kT9XNPSVSAoPNbF34/R/d/Uzg08AOd59uZvsBj5jZAwQz4Z7p7q+Z2UjgMTO7F7ic4Nkt\nDQBmNmsf59zl7u8L9/0l8H/cfYOZzQRuAv6u1G9SpDeUNET29mbuwz7PycBkM/tIuDwcmEAwseE1\nZnYc0E4wffpf9eKct0PHzKjHAD8KpjACYL9eHE8kEUoaItEYcJG7d5mTK+xiGgVMc/c9ZvYCMKjI\nz7fStTu4cJ+d4dd+wPYiSUskE1TTEIlmJTA/nPIaMzvczPYnuOJ4JUwYf0tQnAZ4nWByzJxNwEQz\n28/MhgMnFDtJ+NyFP5rZ2eF5zMymJPOWROLTlYZINN8E6oHHw6mvtxDcubQc+KmZNQHrgGcB3H2b\nmT1iZuuBFe7+WTO7A3gC2AD8todzzQFuNrPPAwMIHjX7u/wdzGw00AQcALSb2SXAxGp62I9kk265\nFRGRyNQ9JSIikSlpiIhIZEoaIiISmZKGiIhEpqQhIiKRKWmIiEhkShoiIhLZ/wJkNWdz+rsZ5wAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb3fceb8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值\n",
    "u = np.mean(X, axis=0)\n",
    "# 方差\n",
    "v = np.std(X, axis=0)\n",
    "\n",
    "X = (X - u) / v\n",
    "\n",
    "# 作图\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Normalization data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 直线初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# X加上偏置项\n",
    "X = np.hstack((np.ones((X.shape[0],1)), X))\n",
    "# 权重初始化\n",
    "w = np.random.randn(3,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "显示初始化直线位置："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEKCAYAAADuEgmxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X2UHHWd7/H3d2aSTEIegCQQIEwG\nMAJJyMNkAoICWZXH3cuKiAayIl7caBAJHOQKlwvhqCB7dAVUCESQFclR2KMiCjEI6hKeNI9geJCw\nQGQgKyRAgISQzMz3/lHdme6enpmqma6u6u7P65w6M1VdU/XtmqS/U7/v7/crc3dERETCqEs6ABER\nqRxKGiIiEpqShoiIhKakISIioSlpiIhIaEoaIiISmpKGiIiEpqQhIiKhKWmIiEhoDUkHUGpjxozx\n5ubmpMMQEakoq1at2uTuY/var+qSRnNzMytXrkw6DBGRimJmG8Lsp+YpEREJTUlDRERCU9IQEZHQ\nqq6mUczOnTtpa2tj+/btSYdSkRobGxk/fjyDBg1KOhQRSVhNJI22tjZGjBhBc3MzZpZ0OBXF3dm8\neTNtbW0ccMABSYcjIgmrieap7du3M3r0aCWMfjAzRo8erbs0EQFqJGkAShgDoGtXW5YsgeZmqKsL\nvi5ZknREkiY10TwlIuEsWQLz5sG2bcH6hg3BOsDcucnFJelRM3caSauvr2f69OlMmTKF008/nW3Z\n/5URfOELX+Dpp58G4Oqrr8577aijjipJnFLbLrusK2FkbdsWbBeBhJOGmf3IzF4zs3U9vD7bzLaY\n2drMckW5YyyVoUOHsnbtWtatW8fgwYO56aabIh/jlltuYdKkSUD3pPHoo4+WJE6pbX/7W7TtUnuS\nvtP4D+DEPvZZ7u7TM8vXyxBT7G26Rx99NM8//zwA3/3ud5kyZQpTpkzhuuuuA2Dr1q384z/+I9Om\nTWPKlCnceeedAMyePZuVK1dyySWX8N577zF9+nTmZtoMhg8fDsBnPvMZ7rvvvl3nOvvss/n5z39O\nR0cHF198MbNmzWLq1KncfPPNpX1TUhWamqJtl9qTaNJw94eAN5KMoVC2TXfDBnDvatMtVeJob29n\n6dKlHHbYYaxatYrbbruNP/3pTzz++OP88Ic/ZM2aNfz2t79l33335YknnmDdunWceGJ+Xr3mmmt2\n3bksKQhszpw5u5LMjh07ePDBBzn55JO59dZbGTVqFCtWrGDFihX88Ic/5MUXXyzNm5KqcdVVMGxY\n/rZhw4LtUlytdRxI+k4jjCPN7AkzW2pmk+M+WVxtutk7g9bWVpqamjjnnHN4+OGHOfXUU9ltt90Y\nPnw4n/zkJ1m+fDmHHXYYDzzwAF/72tdYvnw5o0aNCn2ek046id///ve8//77LF26lGOOOYahQ4dy\n//33c/vttzN9+nSOOOIINm/ezPr16wf2pqTqzJ0LixfDhAlgFnxdvFhF8J7E/UdmGqW999RqYIK7\nv2tmJwN3AxMLdzKzecA8gKYB3kfH1aabvTPI5e5F9/3gBz/IqlWruO+++7j00ks5/vjjueKKcOWc\nxsZGZs+ezbJly7jzzjs544wzdp3r+9//PieccMLA3ohUvblzlSTC6u2PzGq9hqm+03D3t9393cz3\n9wGDzGxMkf0Wu3uru7eOHdvndPC9Kmeb7jHHHMPdd9/Ntm3b2Lp1K7/85S85+uijefXVVxk2bBj/\n8i//wle/+lVWr17d7WcHDRrEzp07ix53zpw53HbbbSxfvnxXkjjhhBNYtGjRrp957rnn2Lp1a+nf\nlEgNqcWOA6m+0zCzccDf3d3N7HCCJLc5znNedVV+P3WIr023paWFs88+m8MPPxwIutTOmDGDZcuW\ncfHFF1NXV8egQYNYtGhRt5+dN28eU6dOpaWlpVtd4/jjj+ess87ilFNOYfDgwbuO/dJLL9HS0oK7\nM3bsWO6+++7SvymRGtLUFDRJFdtetdw9sQX4KbAR2Am0AecAXwK+lHn9POAp4AngceCovo45c+ZM\nL/T0009329abO+5wnzDB3Sz4escdkX68KkW9hiIDUSn/B++4w33YMPegohEsw4aVJt5yXwNgpYf5\n3A6zUyUtpUga0p2uoZRLnB/EcYjjwz2JaxA2aZj3UIytVK2trV74uNdnnnmGQw89NKGIqoOuoZRL\nc3PxJp8JE+Cll8odTTKSuAZmtsrdW/vaL9WFcBGpPbVYXC6U5mugpCFSAyppAFo1j0oP+3tI8zVQ\n0hCpcpU2AK1aR6VH+T2k+RooaYhUuUqbubZaR6VH+T2k+RooaZSJmXHRRRftWv/Od77DlVdeWfLz\naMp0KZTm9vGezJ0bFHw7O4OvafiwHKiov4e0XgMljWIKe5SVoIfZkCFD+MUvfsGmTZsGfKzeaMp0\nKZTm9vFaUi2/ByWNQldeCRde2JUo3IP1Ad4VNDQ0MG/ePK699tpur73++uucdtppzJo1i1mzZvHI\nI4/s2n7cccfR0tLCF7/4RSZMmLAr6XziE59g5syZTJ48mcWLFwNoynQpKs728XPPhYaGoAmloSFY\nL4VKKtyHjTWu30PZr1WYwRyVtAxocF9np/uCBcFImgULiq/302677eZbtmzxCRMm+FtvveXf/va3\nfeHChe7ufsYZZ/jy5cvd3X3Dhg1+yCGHuLv7l7/8Zb/66qvd3X3p0qUO+Ouvv+7u7ps3b3Z3923b\ntvnkyZN906ZNu85TeF5391/84hd+1llnubv7+++/7+PHj/dt27b5zTff7N/4xjfc3X379u0+c+ZM\nf+GFF7rFr8F9lS2OAWjz5+cPPssu8+cPPNZKGdwXNdZS/x5Kea3QiPAukT7wchNFdhlgwnDv+vC+\n/PLL/etf/3pe0hg7dqxPmzZt17Lvvvv622+/7dOmTcv7AN9jjz12JY2FCxf61KlTferUqT5y5Eh/\n7LHH8s5TeN733nvPx48f79u3b/e7777bzzzzTHd3P+2003zixIm7zt3c3OzLli3rFr+ShhSqry+e\nNOrrB3bcCROKH3fChFJEXVpJx1rK84dNGqmesDARZnDttXD99V3brr022F4CF1xwAS0tLXz+85/f\nta2zs5PHHnuMoUOH5u0b/B67++Mf/8gDDzzAY489xrBhw5g9ezbbt2/v9byaMl1KraMj2vawKqlw\nn3SsSZxfNY1Cnqlh5MqtcQzQnnvuyac//WluvfXWXduOP/54fvCDH+xazz534yMf+Qh33XUXAPff\nfz9vvvkmAFu2bGGPPfZg2LBhPPvsszz++OO7flZTpku51NdH2x5WJRWMk441ifMraeTKJozrr4cF\nC4K+bgsWBOslTBwXXXRRXi+q733ve6xcuZKpU6cyadIkbrrpJgAWLlzI/fffT0tLC0uXLmWfffZh\nxIgRnHjiibS3tzN16lQuv/xyPvShD+06VnbK9LlF+ucdf/zxPPTQQ3z84x/PmzJ90qRJtLS0MGXK\nFL74xS/S3t5ekvcp1W3evGjbw0pDwTjp4nZYiZw/TBtWJS0DrmksXJhfw8jWODL1h3Lavn2779y5\n093dH330UZ82bVrZY8hSTUMK3XGHe0NDflt6Q0M6pwWPUjBOurgdVanOj2a57RJ5hlb3/BpG4XqZ\nrF+/nk9/+tN0dnYyePBgbrzxRmbNmlX2OECz3Ep3lTQbbZRYK+l9lVLYWW5VCC+mMEEkkDAAJk6c\nyJo1axI5t0hfki4CRxEl1kp6X0momZpGtd1RlZOunRSTdBE4iiixVtL7SkJNJI3GxkY2b96sD79+\ncHc2b95MY2Nj0qGURVyja+MowsYl7CjvqEXYON5XHAXrpIvbaVcTzVPjx4+nra2N119/PelQKlJj\nYyPjx49POozYZaeuzs5Emp26GgY2WVyU48YVQ1jnnguLFnWtd3R0rd94Y/6+2XguuyxoumlqCj5Y\ni8UZx/uKcswosUbZtxbVRCFcJIy4CqCVVIRtaCg+OK++HgbSEzuO95X0tao2YQvhShoiGXV1xYfi\nmAVDdspx3LhiCKu3Ph8D+aiI430lfa2qjZ4RLhJRXAXQSirCVtIo76SvVa1S0hDJiKuwW0lF2LSM\n8g5zbeO8VpXUcaHswowArKSl2IhwkbDCjq6Nc9Rw0iOM58/vmsG2vn7gU51nxXFt47hWcY4eTzM0\nIlwkPirCxifpa1tJHRdKqSJqGmb2IzN7zczW9fC6mdn3zOx5M3vSzFrKHaNIMRo1HJ+kr61Gj/cu\n6ZrGfwAn9vL6ScDEzDIPWNTLviJloyJsdGHb/qNc2zjqCZXUcSEJiSYNd38IeKOXXf4ZuD3T5PY4\nsLuZ7VOe6ER6lnTButJkB+Jt2BC0/GcH4g2kwB3lmFFUUseFRIQpfMS5AM3Auh5e+w3wkZz1B4HW\n3o6nQriUS9IF60oS9bGkYa5tnI9araSOC6VCpRTCzawZ+I27Tyny2r3At9z94cz6g8D/cfdVBfvN\nI2i+oqmpaeaGYpUpEUmMBvelX0UUwkNoA/bPWR8PvFq4k7svdvdWd28dO3Zs2YITkXA0uK96pD1p\n3AOclelF9SFgi7tvTDookaiiFGzDzjIb1/njEEfbf03WE9IgTBtWXAvwU2AjsJPgruIc4EvAlzKv\nG3AD8N/AX+ijnuGqaUgKRRkANn9+8Xb6gQywS8sAtLgG4lVDPSENqJSaRqlpcJ+kTZQBYHHMMltN\nA9AkPtVS0xCpeFEGgBVLGL1tL/X5RfqipCESsygF2zhmmVXBWEpJSUMkZlddBYMH528bPLh4wTbq\nLLNJzwYrtUdJQ6QMCkuHPZUSP/zhoK6Rq6Eh2F4o7IjouXNh8eKghmEWfF28WI8vlf5RIVwkZnHN\nmqoCt5SSCuEiKRHXrKkqcEsSlDREYhbXrKkqcEsSlDRE+insyO24Zk296ioYNCh/26BBAy9w61Gn\n0qswIwAradGIcCmHqCO345g19Y473AcPzj//4MEDGxVdq486FY0ITzoMqXJxjNyOKo5CuArxtUuF\ncJEYxTFyO6o4CuEqxEtflDSkYiXZnh515HbSjyWN45gqxNcmJQ2pSHE96jOsKCO344r15JOjbQ9D\njzqVPoUpfFTSokJ4bYjzUZ9hzZ/vXl8fnLe+vucieFyxxnXcWnzUqagQnnQYErNKetRnXLFW0jWQ\n9FMhXKpaJbWnxxVrJV0DqR5KGlKR0tCeHra4HVesabgGUoPCtGFV0qKaRu1Isj096sC2uGJVTUFK\nBdU0ROKjgW1SbVTTEImRBrZJrVLSEOkHFaGlVilpiPRDXDPMiqSdkoZIP5n1vi5SjZQ0RPrhsstg\nx478bTt2BNtFqpmSRq5XXik+xFakgArhUqsakg4gNd58E8aPh9GjoaUFZszo+vqBDwQjuEQympqK\nd7lVIVyqXaKfhGZ2opn91cyeN7NLirx+tpm9bmZrM8sXYgumrg5uuAE+8QnYvBmuvRbmzIGDD4ZR\no+Doo2HBAvjxj+HJJ2HnzthCkfSLOhpbj0WVapHYnYaZ1QM3AMcBbcAKM7vH3Z8u2PVOdz8v9oBG\njcp/yPOOHfD007B6NaxZE3y95RbYti14fcgQOOyw/LuSww6DoUNjD1WSN3du8PWyy4ImqaamIGFk\nt+fKTo2e/aeTnRo99zgilSKxEeFmdiRwpbufkFm/FMDdv5Wzz9lAa5SkEeuI8I4OWL8+P5GsWRM0\nbUHwBJ5DD81PJNOnw8iR8cQjFUGjx6UShB0RnmRNYz/g5Zz1NuCIIvudZmbHAM8BF7r7y4U7mNk8\nYB5AU5yNyvX1cMghwXLmmcE2zzxVJzeR/O53cPvtXT/3gQ90JZFsQhk7Nr44JVVUNJdqkmRNo1iv\n9sLbnl8Dze4+FXgA+HGxA7n7YndvdffWseX+MDYL/pT85CfhG9+Ae++FV1+FjRvhvvvgm9+EadNg\n5Uq49FI44QTYay/Yf3845RS48kr41a/g5ZertudWrbfnRx09XuvXS9ItyTuNNmD/nPXxwKu5O7j7\n5pzVHwL/Voa4SmPcODjppGDJevNNWLs2/67k3nu7npgzZkx+r62WFjjooIruuaX2/KDWkXsNoOei\nua6XpF2SNY0GgianjwGvACuAM939qZx99nH3jZnvTwW+5u4f6u24FTfL7datQW+s3ESybl1X76wR\nI4K6SG4iOfRQaKiM3tJqzw8sWRKuaK7rJUkJW9PoMWmY2WEEf93vBywl+MB+M/Pan9398BIEeTJw\nHVAP/MjdrzKzrxPM636PmX0LOAVoB94A5rv7s70ds+KSRjE7dsBTT3UlkdWr4Ykn8ntuTZ2af1eS\n0p5beiRpNLpekpRSJI2HgW8CjwNfAD4PnOLu/21ma9x9RikDLpWqSBrFdHTAc8/l99pavRreeit4\nvb4eJk3KTyQp6Lmlv5yj0fWSpJSi99Rwd/9t5vvvmNkq4Ldm9lm6F6wlbtnuvIceWrznVjaR3H9/\n955bub22ytxzK0p7vuh6Sfr1ljTMzEa5+xYAd/+DmZ0G/BzYsyzRSe+yPbeyvbeyNm4MEkj2buTP\nf4a77up6ffz4/BpJSwvst18s07RGGQQnul6Sfr01T50JvODujxdsbwIud/d/LUN8kVVt89RAvfFG\n0HMrt3nr2We7GtDHjOk+51aF99wSkfAGXNOoVEoaEbz7btBzKzeRFPbcyjZpZRNJBfXcEpHwKmFE\nuCRt+HA46qhgyXr//a45t7KJZPFieO+94PXGxuJzbjU2JvMeRKSsdKchfcv23MrptfX+n9YwZFvQ\nc6udet7ZfxJ7fLRgzq0RI4DwYxREJDlqnpLYLFkC8/7V2eu9l2hhNTNYw6y61RwzYjVDt/y9a8eJ\nE3lpdAu3rprB4ztbWMMMNjOGYcOCmxclDpH0KFnSMLMPAouAvd19iplNJRiv8c3ShFpaShrx63Us\nwWMb8wYlvnzPGvbveGnXPn9jf9YwgxdGtXDhTzJ3JjH13BKR8EqZNP4LuBi4OTugz8zWufuUkkRa\nYkoa8YsyarmuDnb3N5jO2l13JS2s5mD+Sl12uM/Ysd3n3DrwQPXcEimjUhbCh7n7ny3/L8H2fkcm\nFS/Ko06DfffkD3yUP/DRXdsP3f9dnv5ZwZxb//7v3Xtu5SaSQw5Rzy2RhIX5H7jJzA4iMwrczD4F\nbIw1Kkm1KKOWe9r3sm/10HOrcM6tm2/O77mVO+dWSwtMmaKeWyJlFKZ56kBgMXAU8CbwIjDX3Yv8\nrZk8NU+VR5QeUQPqPdXRAX/9a/c5t7ZsCV5vaCg+51am55aIhFOSmoaZ1QGfcve7zGw3oM7d3ylh\nnCWnpFED3OHFF7snkr9nem6ZwcSJ+Ylkxoxg1LuIFFXKQvhD7n5MySKLmZJGDdu4MX9Q4urV+cWX\n/ffvPufWvvuq55YIpU0alwPvAXcCW7Pb3f2NgQYZByUNyfPGG/mTN65ZEzR3eU7PrcI5t9RzS2pQ\nKZPGi0U2u7sf2N/g4qSkIX16993goVaFc261ZzoFjhzZfc4t9dySKleyLrfufkBpQhJJieHD4cMf\nDpasbM+t3OatYj23cp9Nop5bUoPC3GmcVWy7u99ebHvSdKchJdPe3m3OLdas6d5zK7d5a9o09dyS\nilTK5qnv56w2Ah8DVrv7pwYWYjyUNCRW2Z5buYlk9Wp47bXg9WzPrcI6yejRycYt0ofYJiw0s1HA\nT9z9lP4GFycljcpWkTPiunf13Mq9I8ntudXUlN9ra8YM9dySVInzeRrbgIn9+DmRXi1Zkj96fMOG\nYB1SnjjMggSw777wT//UtX3z5uBpibnJ5J57unpu7bVX8Tm3lEgkxcI0T/0asjPLUQdMAv7T3b8W\nc2z9ojuNytXr7LkvlTuamGR7buUmkqee6uq5NWpUMKI9N5EcfLB6bknsSlnTODZntR3Y4O5tA4wv\nNkoalSvK7LlV5f33gy6/uYnkySfze25Nm5afSKZMgSFDko1bqkopm6dOLryrMLN/S+udhlSuKLPn\nVpUhQ2DmzGDJam/Pn3Nr9eqg/W7RouD1hgaYPDm/eUs9t6QMwtxprHb3loJtT7r71Fgj6yfdaVSu\nwpoGoKf85ersLD7nVm7PrQ9+sPucW+q5JSEM+E7DzOYD5wIHmtmTOS+NAB4ZeIhgZicC1wP1wC3u\nfk3B60OA24GZwGbgM+7+UinOLemTTQwV13uqXOrq4KCDguVTmR7vuT23sonk0UfhZz/r+rmmpvxe\nWy0tsM8+KrhLv/R4p5HpWrsH8C3gkpyX3inFvFNmVg88BxwHtAErgDPc/emcfc4Fprr7l8xsDnCq\nu3+mt+PqTkOEoOdW7pxbq1fD+vX5PbcKJ2884AAlkhpW8nEaZrYXweA+ANz9b/0PD8zsSOBKdz8h\ns35p5rjfytlnWWafx8ysAfgfYKz3ErSShkgP3nmn+5xbhT23CufcUs+tmlGyQriZ/S/gu8C+wGvA\nBOAZYPIAY9wPeDlnvQ04oqd93L3dzLYAo4FNAzy3SO0ZMQI+8pFgydq+Pei5lZtIFi0KtgMMHRoU\n2HMTiXpu1bQwf0J8E/gQ8IC7zzCzfwDOKMG5i90HF95BhNkHM5sHzANoqvquNiIl1NgIra3BkpXt\nuZVbJynWc6twzq3hw5N5D1JWYZLGTnffbGZ1Zlbn7n8ws38rwbnbgP1z1scDr/awT1umeWoU0K2e\n4u6LCR5JS2tra7R5UUQkXzYpTJ4Mn/1ssC3bcyu319ZvfgO33Ra8nu25VTjn1p57Jvc+JBZhksZb\nZjYcWA4sMbPXCAb5DdQKYKKZHQC8AswBzizY5x7gc8BjwKeA3/dWzxCRmOT23Dr99GCbO7z6an4i\neeQR+OlPu35uwoTuiUQ9typamHEauxE8ua8OmEvw1/4Sd9884JObnQxcR9Dl9kfufpWZfR1Y6e73\nmFkj8BNgBsEdxhx3f6G3Y6oQLpKwTZu65tzKJpTnnut6fe+9u0/eqJ5biStp7ykzmwBMdPcHzGwY\nUO/u75QgzpJT0hBJoWzPrcI5tzo6gtd33734nFv19cnGXUNK2XvqXwmKzHsCBxH0aLqJ4LkaIiJ9\n663nVm4iufHG7j23chPJ5MnquZWwMM1Ta4HDgT+5+4zMtr+4+2FliC8y3WmIVLD2dnj22fxBiWvX\nwttvB68PGpQ/51a259ZuuyUbdxUo5YSF77v7Dsu0N2Z6MakYLSKl19AQjAOZMiW/59YLL+SPJSns\nuXXwwd3n3FLPrViESRr/ZWb/FxhqZscRzEf163jDEhHJqKuDD3wgWHJ7br3ySn4iefjh/J5bzc3d\nH3K1zz6JvIVqEqZ5qg44BzieYLDdMoLJBVN5t6HmKZEatmlT91mA16/ven3vvbvPudXcrJ5blKD3\nlJk1DXR+qSQoaYhInrffLj7nVm7PrWJzbtVYz61S1DTuBloyB/u5u59WquBERMpm5Eg4+uhgydq+\nHf7yl/xEcsMNwVMUIXiQS+GcW+q5BfSeNHLv1w6MOxARkbJpbIRZs4IlK9tzK3dQ4k9+EnQDhq6e\nW7mDEmuw51ZvScN7+F5EpPrk9tw666xgW7bnVm6N5J574Ec/Cl7P9twqnCpljz2Sex8x662m0QFs\nJbjjGApkH8JpgLv7yLJEGJFqGiISq2zPrdxEsmYNvJzzpIfm5iCBtLbCJZdURKG95A9hqhRKGiKS\niNyeW9lEMmhQUHSvAKUc3CciIn0ZMwaOOy5YsnbsSC6emNQlHYCISNUaPDjpCEpOSUNEREJT0hAR\nkdCUNEREJDQlDRERCU1JQ0REQlPSEBGR0JQ0REQkNCUNEREJTUlDRERCU9IQEZHQlDRERCQ0JQ0R\nEQktkaRhZnua2e/MbH3ma9EnlphZh5mtzSz3lDtOERHJl9SdxiXAg+4+EXgws17Me+4+PbOcUr7w\nRESkmKSSxj8DP858/2PgEwnFISIiESSVNPZ2940Ama979bBfo5mtNLPHzUyJRUQkYbE9uc/MHgDG\nFXnpsgiHaXL3V83sQOD3ZvYXd//vIueaB8wDaGpq6le8IiLSt9iShrt/vKfXzOzvZraPu280s32A\n13o4xquZry+Y2R+BGUC3pOHui4HFEDwjvAThi4hIEUk1T90DfC7z/eeAXxXuYGZ7mNmQzPdjgA8D\nT5ctQhER6SappHENcJyZrQeOy6xjZq1mdktmn0OBlWb2BPAH4Bp3V9IQEUlQbM1TvXH3zcDHimxf\nCXwh8/2jwGFlDk1ERHqhEeEiIhKakoaIiISmpCEiIqEpaUgX997XayUGEemRkoYErrwSLryw60Pa\nPVi/8sraikFEeqWkIcGH81tvwfXXd31oX3hhsP7WW+X5az8NMYhIn8yr7D9ja2urr1y5MukwKk/u\nh3TWggVw7bVgVjsxiNQoM1vl7q197qekIbu4Q13OzWdnZ/k/rNMQg0gNCps01Dwlgexf+bly6wu1\nEoOI9EpJQ/KbhRYsCP66X7Agv75QCzGISJ8SmUZEUsYMdt89v35w7bXBa7vvXp7moTTEICJ9Uk1D\nurjnfzgXrtdKDCI1SDUNia7ww7kWP6w7O3tfLwcNcJQUU9KQ9Eh6cN/s2TBzZlei6OwM1mfPLs/5\nIflrINIHJQ1Jh6QH93V2wpYtsHZtV+KYOTNY37KlPHccSV8DkRBU05D0SHpwX26iyJo+HVatyh87\nEqekr4HULA3uk8qU9OC+zk6or+9a7+goX8LISvoaSE1SIVwCUYqqcRWBOzp6X89yhwsuyN92wQXl\na5bJ3mnkyq1xlIMGOErKKWlUsyhF1biKwM3NMG5cV6Lo6AjWm5vz93OHI4+E730Pzj8/OP/55wfr\nRx5ZnppGtmlq+vQgzunT82sccdMAR6kAShrVKkpRNa4icEcHbN0KmzZ1JY5x44L1rVt7vuNIQl0d\njBqVX8NYtSpYHzWqPE1UPQ1wXLBAAxwlNVTTqGZRiqpxFYFzE0XWmDHwP/+TXzvIxnvBBcHdRdb5\n58N115XvA7Ozs3s9IYmahgY4SpmpEC6BKEXVuIrAHR3QkDNjTXt794TRn3hFpGRUCK92YQrcUYqq\nUYvAYYvm2TuNXLk1jsJ4oxTCw8aQhs4AUWhEuKSYkkYlClPgjlJUjVoEDls07+iAkSODpqkxY4I7\njDFjgvWRI/MTh3tQOyhWCB+LV6m7AAAKN0lEQVQ1qvsHZ9gY0tAZIAqNCJeUU9KoNGEL3FGKqlGK\nwFGK5nV1Xc1Qc+YE63PmBOv19d2P294efL9kSbC+ZEmw3t6ef9ywMaShM0AUGhEulcDdq2qZOXOm\nV73OTvcFC9yDj5FgWbAg2F5s397Wc3V09L6eu3369PzzT59efP/OTvfzzsvf97zzisfR3u4+enT+\nvqNHB9v7G0OUaxXlfcUlSrwiJQSs9BCfsYkUws3sdOBK4FDgcHcvWrk2sxOB64F64BZ3v6avY6eu\nEB5XT5goxeWwovQcam+HQYO61nfuzI+nv7Hu3AmDB3et79iRf57+HDeuzgBx/W7VGUASkPZC+Drg\nk8BDPe1gZvXADcBJwCTgDDObVJ7wSiSu9uljjy1eXD722P4fM0p7/rHHwogR+dtGjCh+/iixHn00\nDB2av23o0GB7oSuuKH7cK67I3xZXZ4C4frdR4hVJQpjbkbgW4I9Aaw+vHQksy1m/FLi0r2Ompnkq\nt5kh27xQuN4f7e3uY8YExxkzpvh6VLnNMtnmmML1rJ073Rsbg9caG4uv9yfWHTvc6+uD1+rri69H\nPW6U30GUaxDX7zau44qEQMjmqTQ/7nU/4OWc9TbgiGI7mtk8YB5AU1NT/JGFkfu40uuv7xpgN9AZ\nS+vrYf58WLQo6IWUbZ4ZMybY3p8mqmzhO1v4zR6j2OC+hgaYNQtWrIDt27uajhobg+25zUVRYh00\nCPbbD155JWgSyjZR1dcH23ObqOrrYdIkePrp7sedNKnruFEeIdtTZ4CZM7t3Bojrd6tH3koFiK2m\nYWYPAOOKvHSZu/8qs88fga96kZpGpu5xgrt/IbP+WYL6x1d6O28qaxpxtE/HVdMI255faTWNsLWH\nKHWduH63cdVKRHqReE3D3T/u7lOKLL8KeYg2YP+c9fHAq6WPNEbuwV+NuRYs6Ll9unB7T/t1dkJr\nwe+2tbV423uUY4Ztz+/sDO4qcs2a1fO+YWPt7ITDD8/fdvjhPb+viy7K33bRRcXfX5TH2Ibd1yMO\nRIxCj92VNAvThhXXQu81jQbgBeAAYDDwBDC5r2Omqqax335Be/RXvhKsf+Urwfp++3Vvn164ML/d\nOtuevXBh/n5R2t7jOGbUfceNK77vuHH9P25cbf9hr1dnp/sRRwTnO//8YP3884P1I45Q7UEqEiFr\nGon0njKzU82sjaDYfa+ZLcts39fM7gNw93bgPGAZ8Axwl7s/lUS8/eLeNeJ5+fJgffnyYL2jI/8v\nUo8wqCvsQLw4jhl1X7Ou5qKjjw7Wsz2h6uv7P8Awjtlgo1wvkVoWJrNU0pKaOw336IPgogzqCjMQ\nL45jRt039w4ru2TvvAYaQ5SBi2FEHTSZvbvILtm7DpEKRJoH98UpdYXwqIPFSl1YjeOYlRhDWFFi\nraT3JdKHxAvhQrTicrY5JNdAB3XFccz+xJDkI1yjiHK90nBts3H0ti5SamFuRyppSU3zVNKF3TQM\nFKukgnGU65WGa+sevnAvEgJVMLivskUdLFbqQV0aKBZNlOuVhmvrOYV7CM6fOxW+a2yHxEM1jbhF\nHSxW6kFdcRwz6vmTfoRrFFGuVxqubdjH+Yr0QY97lfRQwTg+urZSIiqEx0nFx/DSUjCuRrq2kgAl\njaj0OM7wcptP+nrkrESjaysJUSE8ChUfo0lDwbha6dpKQlTTiErFx+iSLhhXM11bKREVwuOUhuKj\nPixEpIRUCI9LGoqPqquISEKUNKJIQ/Ext66i2VhFpMxUCI8iDcXHuB41KiISgmoa/ZGGekIa6ioi\nUjVU04hT0o/jTENdRURqkpJGpUlDXUVEapZqGpUmDXUVEalZqmlUqjTUVUSkaqimUe2SrquISE1S\n0hARkdCUNEREJDQlDRERCU1JQ0REQlPSEBGR0JQ0REQktKobp2FmrwMbBnCIMcCmEoVTLXRNutM1\n6U7XpLtKuiYT3H1sXztVXdIYKDNbGWaASy3RNelO16Q7XZPuqvGaqHlKRERCU9IQEZHQlDS6W5x0\nACmka9Kdrkl3uibdVd01UU1DRERC052GiIiEpqRRwMy+bWbPmtmTZvZLM9s96ZjSwMxON7OnzKzT\nzKqqN0gUZnaimf3VzJ43s0uSjicNzOxHZvaama1LOpa0MLP9zewPZvZM5v/NgqRjKhUlje5+B0xx\n96nAc8ClCceTFuuATwIPJR1IUsysHrgBOAmYBJxhZpOSjSoV/gM4MekgUqYduMjdDwU+BHy5Wv6t\nKGkUcPf73b09s/o4MD7JeNLC3Z9x978mHUfCDgeed/cX3H0H8DPgnxOOKXHu/hDwRtJxpIm7b3T3\n1Znv3wGeAfZLNqrSUNLo3f8GliYdhKTGfsDLOettVMkHgcTHzJqBGcCfko2kNGryGeFm9gAwrshL\nl7n7rzL7XEZwi7mknLElKcx1qXHFHo+o7ofSIzMbDvwcuMDd3046nlKoyaTh7h/v7XUz+xzwT8DH\nvIb6JPd1XYQ2YP+c9fHAqwnFIilnZoMIEsYSd/9F0vGUipqnCpjZicDXgFPcfVvS8UiqrAAmmtkB\nZjYYmAPck3BMkkJmZsCtwDPu/t2k4yklJY3ufgCMAH5nZmvN7KakA0oDMzvVzNqAI4F7zWxZ0jGV\nW6aDxHnAMoLC5l3u/lSyUSXPzH4KPAYcbGZtZnZO0jGlwIeBzwIfzXyOrDWzk5MOqhQ0IlxERELT\nnYaIiISmpCEiIqEpaYiISGhKGiIiEpqShoiIhKakIVLAzDpyukmuzUwDEfUYu5vZuaWPbtfxDzGz\nx8zsfTP7alznESmkLrciBczsXXcfPsBjNAO/cfcpEX+u3t07Quy3FzAB+ATwprt/pz9xikSlOw2R\nEMysPvOslRWZZ618MbN9uJk9aGarzewvZpad9fYa4KDMncq3zWy2mf0m53g/MLOzM9+/ZGZXmNnD\nwOlmdpCZ/dbMVpnZcjM7pDAed3/N3VcAO2N/8yI5anLuKZE+DDWztZnvX3T3U4FzgC3uPsvMhgCP\nmNn9BLPenurub5vZGOBxM7sHuITguSzTAcxsdh/n3O7uH8ns+yDwJXdfb2ZHADcCHy31mxTpDyUN\nke7ey37Y5zgemGpmn8qsjwImEkxieLWZHQN0EkyVvnc/znkn7JoV9SjgP4PpiwAY0o/jicRCSUMk\nHAO+4u55c25lmpjGAjPdfaeZvQQ0Fvn5dvKbgwv32Zr5Wge8VSRpiaSCahoi4SwD5memu8bMPmhm\nuxHccbyWSRj/QFCcBniHYOLLrA3AJDMbYmajgI8VO0nmmQsvmtnpmfOYmU2L5y2JRKc7DZFwbgGa\ngdWZaa9fJ+i5tAT4tZmtBNYCzwK4+2Yze8TM1gFL3f1iM7sLeBJYD6zp5VxzgUVm9v+AQQSPlX0i\ndwczGwesBEYCnWZ2ATCpWh70I+mlLrciIhKamqdERCQ0JQ0REQlNSUNEREJT0hARkdCUNEREJDQl\nDRERCU1JQ0REQlPSEBGR0P4/4Dr5/QEAeq8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa3070b8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上图可见，一般随机生成的分类线，错误率很高。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 计算scores，更新权重"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，计算scores，得分函数与阈值0做比较，大于零则$\\hat y=1$，小于零则$\\hat y=-1$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "s = np.dot(X, w)\n",
    "y_pred = np.ones_like(y)    # 预测输出初始化\n",
    "loc_n = np.where(s < 0)[0]    # 大于零索引下标\n",
    "y_pred[loc_n] = -1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接着，从分类错误的样本中选择一个，使用PLA更新权重系数$w$。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# 第一个分类错误的点\n",
    "t = np.where(y != y_pred)[0][0]\n",
    "# 更新权重w\n",
    "w += y[t] * X[t, :].reshape((3,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 迭代更新训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "更新权重$w$是个迭代过程，只要存在分类错误的样本，就不断进行更新，直至所有的样本都分类正确。（注意，前提是正负样本完全可分）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第 0次更新，分类错误的点个数： 0\n"
     ]
    }
   ],
   "source": [
    "for i in range(100):\n",
    "    s = np.dot(X, w)\n",
    "    y_pred = np.ones_like(y)\n",
    "    loc_n = np.where(s < 0)[0]\n",
    "    y_pred[loc_n] = -1\n",
    "    num_fault = len(np.where(y != y_pred)[0])\n",
    "    print('第%2d次更新，分类错误的点个数：%2d' % (i, num_fault))\n",
    "    if num_fault == 0:\n",
    "        break\n",
    "    else:\n",
    "        t = np.where(y != y_pred)[0][0]\n",
    "        w += y[t] * X[t, :].reshape((3,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代完毕后，得到更新后的权重系数$w$，绘制此时的分类直线是什么样子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY0AAAEKCAYAAADuEgmxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X+UFPWZ7/H3w8AIyA8VMPwYB0yW\nbEQcEAbU+CMkUURzV2OMCYZd464uhvgDcow3ermKx7267klOjNkkKGqym5Xd1d0YY7ISjCZZiUoW\nUEyIZqNrJBkgCqhowBFm5rl/VDfT3dPTUzXT1VXd/XmdU2emqqurnv6O9kN9n/p+y9wdERGRMAYl\nHYCIiFQPJQ0REQlNSUNEREJT0hARkdCUNEREJDQlDRERCU1JQ0REQlPSEBGR0JQ0REQktMFJB1Bu\nY8eO9SlTpiQdhohIVdm0adMudx/X1341lzSmTJnCxo0bkw5DRKSqmNnWMPupe0pEREJT0hARkdCU\nNEREJLSaq2kUc+DAAdra2mhvb086lKo0dOhQmpqaGDJkSNKhiEjC6iJptLW1MXLkSKZMmYKZJR1O\nVXF3du/eTVtbG0cffXTS4YhIwuqie6q9vZ0xY8YoYfSDmTFmzBhdpYkIUCdJA1DCGAC1XX1ZvRqm\nTIFBg4Kfq1cnHZGkSV10T4lIOKtXw+LFsG9fsL51a7AOsGhRcnFJetTNlUbSGhoamDlzJtOnT+eC\nCy5gX/b/ygguvfRSnnvuOQBuueWWvNfe//73lyVOqW/Ll3cnjKx9+4LtIpBw0jCzb5rZq2a2pZfX\n55nZHjPbnFluqHSM5TJs2DA2b97Mli1baGxs5I477oh8jLvvvptp06YBPZPGk08+WZY4pb797nfR\ntkv9SfpK4x+ABX3ss87dZ2aWmyoQU+x9uqeeeiovvvgiAF/+8peZPn0606dP5ytf+QoAe/fu5SMf\n+QgzZsxg+vTp3HfffQDMmzePjRs3cu211/L2228zc+ZMFmX6DEaMGAHAJz/5SR5++OGD57r44ov5\nzne+Q2dnJ9dccw1z5syhpaWFO++8s7wfSmpCc3O07VJ/Ek0a7v448FqSMRTK9ulu3Qru3X265Uoc\nHR0drFmzhuOOO45NmzbxrW99i5///OesX7+eu+66i2eeeYYf/vCHTJw4kWeffZYtW7awYEF+Xr31\n1lsPXrmsLghs4cKFB5PM/v37eeyxxzj77LO55557GD16NBs2bGDDhg3cdddd/Pa3vy3Ph5KacfPN\nMHx4/rbhw4PtUly93TiQ9JVGGCeZ2bNmtsbMjo37ZHH16WavDFpbW2lubuaSSy7hZz/7Geeddx6H\nHnooI0aM4GMf+xjr1q3juOOO49FHH+ULX/gC69atY/To0aHPc9ZZZ/HjH/+Yd955hzVr1nDaaacx\nbNgwHnnkEb797W8zc+ZMTjjhBHbv3s0LL7wwsA8lNWfRIli1CiZPBrPg56pVKoL3Ju5/ZKZR2u+e\nehqY7O5/NLOzgQeBqYU7mdliYDFA8wCvo+Pq081eGeRy96L7vve972XTpk08/PDDXHfddcyfP58b\nbghXzhk6dCjz5s1j7dq13HfffVx44YUHz/X3f//3nHnmmQP7IFLzFi1Skgir1D8ya7UNU32l4e5v\nuvsfM78/DAwxs7FF9lvl7q3u3jpuXJ/TwZdUyT7d0047jQcffJB9+/axd+9evvvd73Lqqaeyfft2\nhg8fzp//+Z/z+c9/nqeffrrHe4cMGcKBAweKHnfhwoV861vfYt26dQeTxJlnnsnKlSsPvuc3v/kN\ne/fuLf+HEqkj9XjjQKqvNMxsPPCKu7uZzSVIcrvjPOfNN+ffpw7x9enOmjWLiy++mLlz5wLBLbXH\nH388a9eu5ZprrmHQoEEMGTKElStX9njv4sWLaWlpYdasWT3qGvPnz+eiiy7inHPOobGx8eCxX375\nZWbNmoW7M27cOB588MHyfyiROtLcHHRJFdtes9w9sQX4F2AHcABoAy4BPgN8JvP6FcCvgGeB9cD7\n+zrm7NmzvdBzzz3XY1sp997rPnmyu1nw8957I729JkVtQ5GBqJb/B++91334cPegohEsw4eXJ95K\ntwGw0cN8b4fZqZqWciQN6UltKJUS5xdxHOL4ck+iDcImDfNeirHVqrW11Qsf9/r8889zzDHHJBRR\nbVAbSqVMmVK8y2fyZHj55UpHk4wk2sDMNrl7a1/7pboQLiL1px6Ly4XS3AZKGiJ1oJoGoNXyqPSw\nf4c0t4GShkiNq7YBaLU6Kj3K3yHNbaCkIVLjqm3m2lodlR7l75DmNlDSqBAz4+qrrz64/qUvfYkb\nb7yx7OfRlOlSKM39471ZtCgo+HZ1BT/T8GU5UFH/DmltAyWNYgrvKCvDHWaHHHIIDzzwALt27Rrw\nsUrRlOlSKM394/WkVv4OShqFbrwRPve57kThHqwP8Kpg8ODBLF68mNtuu63Hazt37uT8889nzpw5\nzJkzhyeeeOLg9jPOOINZs2Zx2WWXMXny5INJ56Mf/SizZ8/m2GOPZdWqVQCaMl2KirN//LOfhcGD\ngy6UwYOD9XKopsJ92Fjj+jtUvK3CDOaopmVAg/u6utyXLg1G0ixdWny9nw499FDfs2ePT5482d94\n4w3/4he/6CtWrHB39wsvvNDXrVvn7u5bt271973vfe7ufvnll/stt9zi7u5r1qxxwHfu3Onu7rt3\n73Z393379vmxxx7ru3btOniewvO6uz/wwAN+0UUXubv7O++8401NTb5v3z6/8847/W/+5m/c3b29\nvd1nz57tL730Uo/4NbivusUxAG3JkvzBZ9llyZKBx1otg/uixlruv0M52wqNCO8W6QsvN1FklwEm\nDPfuL+/rr7/eb7rpprykMW7cOJ8xY8bBZeLEif7mm2/6jBkz8r7ADz/88INJY8WKFd7S0uItLS0+\natQof+qpp/LOU3jet99+25uamry9vd0ffPBB/9SnPuXu7ueff75PnTr14LmnTJnia9eu7RG/koYU\namgonjQaGgZ23MmTix938uRyRF1eScdazvOHTRqpnrAwEWZw221w++3d2267LdheBsuWLWPWrFn8\n5V/+5cFtXV1dPPXUUwwbNixv3+Dv2NNPf/pTHn30UZ566imGDx/OvHnzaG9vL3leTZku5dbZGW17\nWNVUuE861iTOr5pGIc/UMHLl1jgG6IgjjuATn/gE99xzz8Ft8+fP52tf+9rB9exzN0455RTuv/9+\nAB555BFef/11APbs2cPhhx/O8OHD+fWvf8369esPvldTpkulNDRE2x5WNRWMk441ifMraeTKJozb\nb4elS4N73ZYuDdbLmDiuvvrqvLuovvrVr7Jx40ZaWlqYNm0ad9xxBwArVqzgkUceYdasWaxZs4YJ\nEyYwcuRIFixYQEdHBy0tLVx//fWceOKJB4+VnTJ9UZH78+bPn8/jjz/O6aefnjdl+rRp05g1axbT\np0/nsssuo6OjoyyfU2rb4sXRtoeVhoJx0sXtsBI5f5g+rGpaBlzTWLEiv4aRrXFk6g+V1N7e7gcO\nHHB39yeffNJnzJhR8RiyVNOQQvfe6z54cH5f+uDB6ZwWPErBOOnidlTlOj+a5bZb5Bla3fNrGIXr\nFfLCCy/wiU98gq6uLhobG/nGN77BnDlzKh4HaJZb6amaZqONEms1fa5yCjvLrQrhxRQmiAQSBsDU\nqVN55plnEjm3SF+SLgJHESXWavpcSaibmkatXVFVktpOikm6CBxFlFir6XMloS6SxtChQ9m9e7e+\n/PrB3dm9ezdDhw5NOpSKiGt0bRxF2LiEHeUdtQgbx+eKo2CddHE77eqie6qpqYm2tjZ27tyZdChV\naejQoTQ1NSUdRuyyU1dnZyLNTl0NA5ssLspx44ohrM9+Flau7F7v7Oxe/8Y38vfNxrN8edB109wc\nfLEWizOOzxXlmFFijbJvPaqLQrhIGHEVQKupCDt4cPHBeQ0NMJA7seP4XEm3Va0JWwhX0hDJGDSo\n+FAcs2DITiWOG1cMYZW652MgXxVxfK6k26rW6BnhIhHFVQCtpiJsNY3yTrqt6pWShkhGXIXdairC\npmWUd5i2jbOtqunGhYoLMwKwmpZiI8JFwgo7ujbOUcNJjzBesqR7BtuGhoFPdZ4VR9vG0VZxjh5P\nMzQiXCQ+KsLGJ+m2raYbF8qpKmoaZvZNM3vVzLb08rqZ2VfN7EUz+4WZzap0jCLFaNRwfJJuW40e\nLy3pmsY/AAtKvH4WMDWzLAZWlthXpGJUhI0ubN9/lLaNo55QTTcuJCHRpOHujwOvldjlXODbmS63\n9cBhZjahMtGJ9C7pgnW1yQ7E27o16PnPDsQbSIE7yjGjqKYbFxIRpvAR5wJMAbb08toPgFNy1h8D\nWksdT4VwqZSkC9bVJOpjScO0bZyPWq2mGxfKhWophJvZFOAH7j69yGv/Afytu/8ss/4Y8L/dfVPB\nfosJuq9obm6evbVYZUpEEqPBfelXFYXwENqAo3LWm4DthTu5+yp3b3X31nHjxlUsOBEJR4P7akfa\nk8ZDwEWZu6hOBPa4+46kgxKJKkrBNuwss3GdPw5x9P3XZT0hDcL0YcW1AP8C7AAOEFxVXAJ8BvhM\n5nUDvg78D/BL+qhnuGoakkJRBoAtWVK8n34gA+zSMgAtroF4tVBPSAOqpaZRbhrcJ2kTZQBYHLPM\n1tIANIlPrdQ0RKpelAFgxRJGqe3lPr9IX5Q0RGIWpWAbxyyzKhhLOSlpiMTs5puhsTF/W2Nj8YJt\n1Flmk54NVuqPkoZIBRSWDnsrJZ58clDXyDV4cLC9UNgR0YsWwapVQQ3DLPi5apUeXyr9o0K4SMzi\nmjVVBW4pJxXCRVIirllTVeCWJChpiMQsrllTVeCWJChpiPRT2JHbcc2aevPNMGRI/rYhQwZe4Naj\nTqWkMCMAq2nRiHCphKgjt+OYNfXee90bG/PP39g4sFHR9fqoU9GI8KTDkBoXx8jtqOIohKsQX79U\nCBeJURwjt6OKoxCuQrz0RUlDqlaS/elRR24n/VjSOI6pQnx9UtKQqhTXoz7DijJyO65Yzz472vYw\n9KhT6VOYwkc1LSqE14c4H/UZ1pIl7g0NwXkbGnovgscVa1zHrcdHnYoK4UmHITGrpkd9xhVrNbWB\npJ8K4VLTqqk/Pa5Yq6kNpHYoaUhVSkN/etjidlyxpqENpA6F6cOqpkU1jfqRZH961IFtccWqmoKU\nC6ppiMRHA9uk1qimIRIjDWyTeqWkIdIPKkJLvVLSEOmHuGaYFUk7JQ2RfjIrvS5Si5Q0RPph+XLY\nvz9/2/79wXaRWqakIdIPKoRLvVLSEOkHFcKlXiWaNMxsgZn9t5m9aGbXFnn9YjPbaWabM8ulScQp\nUijqaGw9FlVqxeCkTmxmDcDXgTOANmCDmT3k7s8V7Hqfu19R8QBFSli0KPi5fHnQJdXcHCSM7PZc\n2anR9+0L1rNTo+ceR6RaJHmlMRd40d1fcvf9wL8C5yYYj0gkixYFo7+7uoKfvSWA5cu7E0bWvn0q\nmkt1SjJpTAJ+n7PeltlW6Hwz+4WZ/buZHVXsQGa22Mw2mtnGnTt3xhGrSL+paC61JMmkUeyu9sKJ\nsL4PTHH3FuBR4B+LHcjdV7l7q7u3jhs3rsxhykDVe39+1KJ5vbeXpFuSSaMNyL1yaAK25+7g7rvd\n/Z3M6l3A7ArFJmWS9GNZ0yBK0VztJWmXZNLYAEw1s6PNrBFYCDyUu4OZTchZPQd4voLxSRmoPz+o\ndaxaFcyAaxb8XLWqeA1E7SVp1+vU6GZ2HMG/7icBa4AvuPvrmdf+y93nDvjkZmcDXwEagG+6+81m\ndhPBvO4PmdnfEiSLDuA1YIm7/7rUMTU1errokaTRqL0kKWGnRi91y+1K4EZgPXAp8DMzO8fd/wcY\nUuJ9obn7w8DDBdtuyPn9OuC6cpxLktHcXPy5ExoEV5zaS9KuVPfUCHf/obu/4e5fAq4AfmhmJ9Kz\nYC1SlB5JGo3aS9KuVNIwMxudXXH3nwDnA/8ETI47MKkNUfrzRe0l6VeqpvEp4CV3X1+wvRm43t3/\nugLxRaaahohIdAOuabj7P/ey/XdAKhOGiIjES7PciohIaEoa0i9RRi1rhLNI7UhsllupXlFmbdUM\nryK1pc8rDTN7r5k9ZmZbMustZvZ/4w9N0irKqGWNcBapLWG6p+4iGGB3AMDdf0Ew5YfUqSiztmqG\nV5HaEiZpDHf3/yrY1hFHMFIdoszaqseiitSWMEljl5m9h8wocDP7OLAj1qgk1aKMWtYIZ5HaEiZp\nXA7cCbzPzLYBy4DPxBqVpFqUUcsa4SxSW3odEQ5gZoOAj7v7/WZ2KDDI3d+qWHT9oBHhIiLRhR0R\nXvJKw927CCYqxN33pj1hiIhIvMJ0T/3IzD5vZkeZ2RHZJfbIREQkdcIM7vurzM/Lc7Y58O7yhyMi\nImnWZ9Jw96MrEYiIiKRfn0nDzC4qtt3dv13+cEREJM3CdE/Nyfl9KPBh4GlASUNEpM70WQh39ytz\nlr8Gjgca4w9N6pFmxBVJt/7McrsPmFruQEQ0I65I+oWpaXyfzBQiBFcm04B/izMoqU+lZsRV0hBJ\nhzBXGl/K+b0D2OrubTHFI3VMM+KKpF+YwX1nu/t/ZpYn3L3NzP4u9sik7mhGXJH0C5M0ziiy7axy\nByKiGXGlZnR2wvbt8PzzSUdSdr12T5nZEuCzwLvN7Bc5L40EnijHyc1sAXA70ADc7e63Frx+CMGt\nvbOB3cAn3f3lcpxb0idbt1i+POiSam4OEobqGZIq+/bBtm3dS1tb/vq2bbBjR5A4Jk+Gl19OOuKy\nKlXT+GdgDfC3wLU5299y99cGemIzawC+TnAl0wZsMLOH3P25nN0uAV539z8xs4XA3wGfHOi5Jb0W\nLVKSkIS4w65dvSeC7LY33uj53lGjYNKkYPnwh4OfTU1B0qgxvSYNd98D7AEuBDCzIwkG940wsxHu\nPtDy5FzgRXd/KXP8fwXOBXKTxrnAjZnf/x34mpmZl5rPXUSk0DvvBN1FvSWCbduC1/fvz3+fGYwf\nHySBP/kTmDevOznkLiNHJvKxkhDmlts/A74MTAReBSYDzwPHDvDck4Df56y3ASf0to+7d5jZHmAM\nsGuA5xaRWuAe/Mu/r+6inTt7vnfYsO4rgpNPzk8CTU3Bz/HjYXB/hrPVrjCt8f+AE4FH3f14M/sg\nmauPAbIi2wqvIMLsg5ktBhYDNOtWG5Ha0NEBf/hD74kgu/3tt3u+d9y47gQwd25+Isguhx0WXElI\nJGGSxgF3321mg8xskLv/pEy33LYBR+WsNwHbe9mnzcwGA6OBHvUUd18FrILgyX1liE1E4vTHP5ZO\nBNu2wSuvQFdX/vsaG2HixOBL//jj4c/+rGdX0cSJcMghyXyuOhAmabxhZiOAdcBqM3uVYJDfQG0A\npprZ0cA2YCHwqYJ9HgI+DTwFfBz4seoZIinW1QWvvlq6drBtG7z5Zs/3HnZY9xVBS0vPZNDUBGPG\nBBOTSWLCJI1zgbeBZcAign/t3zTQE2dqFFcAawluuf2mu//KzG4CNrr7Q8A9wD+Z2YsEVxgLB3pe\nEemn9va+awfbtwfdSrkaGmDChOCL/5hj4PTTi3cXFQ7SkVSyMP9wN7PJwFR3f9TMhgMNaX1eeGtr\nq2/cuDHpMESqhzu89lrf3UWvFbnTfsSI4gXk3OVd7woSh6SamW1y99a+9gtz99RfExSZjwDeQ3BH\n0x0Ez9UQkTTbvz8YaNbXrabt7fnvM4Mjjwy+9I8+Gk45pXh30ahRyXwuSUyY7qnLCcZU/BzA3V/I\njNkQkaS4B3WBvrqLXn012DfXIYd0XxGccELxK4UJE2DIkGQ+m6RamKTxjrvvt8ytaZm7mFSMFolL\nZ2dw51BfI5P37u353jFjuhPArFnFu4yOOEK3mkq/hUka/2lm/wcYZmZnEMxH9f14wxKpUXv3lq4b\nbNsWjE3o7Mx/3+DB3beatrTA2WcXv9V02LBkPpfUjTBJ41qCOaB+CVwGPAzcHWdQIlWnq6t73qJS\n3UWl5i1qaoJp04p3F40bp1tNJRVKzXLb7O6/c/cu4K7MIlJ/cuct6u0Oo23b4MCB/PcNGtQ9b9HU\nqd3zFhV2F40YkcjHEumPUlcaDwKzAMzsO+5+fmVCEqmQ7LxFfd1quqvIVGfDh3d/6Z98cvFbTTVv\nkdSgUv9F51bK3h13ICJl1dHR962m27aVnrfoqKPgxBOLdxeNHq1istSlUknDe/ldJFlvvVW6bpAt\nJhfeatrY2P3lP3s2nHNOz+6iCRM0b5FICaWSxgwze5PgimNY5ncy6+7uGtUj5ZWdt6iv7qK3ikxG\ncPjh3V/8LS3Fu4vGjtXVgcgAlXoIk8b9S/m8/Xbft5ru2FF63qJjj4X584s/BEfzFolUhKp0MjDu\nsHt337ea9jZvUfaK4IMfLF47OPJIzVskkiJKGtK73HmLehuZvH17cEtqLrNgkrrCeYsKu4w0b5FI\n1VHSqEfZeYtKTVGRnbeo0NCh3V/6J55YvHageYtEapaSRq3p7Ox+RGap7qJS8xY1NUFra/HuosMP\nVzFZpI4paVST3HmLervDaMeOno/IHDKke96iGTO65y3KvUqYODG4ihARKUFJIw2y8xb1davpnj09\n3zt6dPcX/7RpxbuLNG+RiJSJkkbc2tu75y0q9RCcUvMW/emfwoc+1LO7aOJEzVskIhWlpNFf7vD6\n6313F/U2b1H2iuDUU4s/KvNd79K8RSKSOvpWKubAgaCYXKq7aPv24vMWZR+RmZ23qFh3keYtEpEq\npaSR9cor8JGPBEnhlVdKz1s0Z07xUckTJwb7iYjUKCWNrFGjgoLxzJnFbzUdM0ZXByJS95Q0soYN\ngzVrko5CRCTVdB+miIiEpqQhIiKhJZI0zOwIM/uRmb2Q+Xl4L/t1mtnmzPJQpeMUEZF8SV1pXAs8\n5u5Tgccy68W87e4zM8s5lQtPRESKSSppnAv8Y+b3fwQ+mlAcIiISQVJJ413uvgMg8/PIXvYbamYb\nzWy9mSmxiIgkLLZbbs3sUWB8kZeWRzhMs7tvN7N3Az82s1+6+/8UOddiYDFAc3Nzv+IVEZG+xZY0\n3P303l4zs1fMbIK77zCzCUCRp/2Au2/P/HzJzH4KHA/0SBruvgpYBdDa2uqFr4uISHkk1T31EPDp\nzO+fBr5XuIOZHW5mh2R+HwucDDxXsQhFRKSHpJLGrcAZZvYCcEZmHTNrNbO7M/scA2w0s2eBnwC3\nuruShohIghKZRsTddwMfLrJ9I3Bp5vcngeMqHJqIiJSgEeEiIhKakoaIiISmpCEiIqEpaUi3wgdP\nFa7XSwwi0islDQnceCN87nPdX9LuwfqNN9ZXDCJSkpKGBF/Ob7wBt9/e/aX9uc8F62+8UZl/7ach\nBhHpk3mN/c/Y2trqGzduTDqM6pP7JZ21dCncdlvlHnObhhhE6pSZbXL31j73U9KQg9xhUM7FZ1dX\n5b+s0xCDSB0KmzTUPSWB7L/yc+XWF+olBhEpSUlD8ruFli4N/nW/dGl+faEeYhCRPiUyjYikjBkc\ndlh+/eC224LXDjusMt1DaYhBRPqkmoZ0c8//ci5cr5cYROqQahoSXeGXcz1+WXd1lV6vBA1wlBRT\n0pD0SHpw37x5MHt2d6Lo6grW582rzPkh+TYQ6YOShqRD0oP7urpgzx7YvLk7ccyeHazv2VOZK46k\n20AkBNU0JD2SHtyXmyiyZs6ETZvyx47EKek2kLqlwX1SnZIe3NfVBQ0N3eudnZVLGFlJt4HUJRXC\nJRClqBpXEbizs/R6ljssW5a/bdmyynXLZK80cuXWOCpBAxwl5ZQ0almUompcReApU2D8+O5E0dkZ\nrE+Zkr+fO5x0Enz1q3DVVcH5r7oqWD/ppMrUNLJdUzNnBnHOnJlf44ibBjhKFVDSqFVRiqpxFYE7\nO2HvXti1qztxjB8frO/d2/sVRxIGDYLRo/NrGJs2BeujR1emi6q3AY5Ll2qAo6SGahq1LEpRNa4i\ncG6iyBo7Fv7wh/zaQTbeZcuCq4usq66Cr3ylcl+YXV096wlJ1DQ0wFEqTIVwCUQpqsZVBO7shME5\nM9Z0dPRMGP2JV0TKRoXwWhemwB2lqBq1CBy2aJ690siVW+MojDdKITxsDGm4GSAKjQiXFFPSqEZh\nCtxRiqpRi8Bhi+adnTBqVNA1NXZscIUxdmywPmpUfuJwD2oHxQrho0f3/OIMG0MabgaIQiPCJeWU\nNKpN2AJ3lKJqlCJwlKL5oEHd3VALFwbrCxcG6w0NPY/b0RH8vnp1sL56dbDe0ZF/3LAxpOFmgCg0\nIlyqgbvX1DJ79myveV1d7kuXugdfI8GydGmwvdi+pdZzdXaWXs/dPnNm/vlnziy+f1eX+xVX5O97\nxRXF4+jocB8zJn/fMWOC7f2NIUpbRflccYkSr0gZARs9xHdsIoVwM7sAuBE4Bpjr7kUr12a2ALgd\naADudvdb+zp26grhcd0JE6W4HFaUO4c6OmDIkO71Awfy4+lvrAcOQGNj9/r+/fnn6c9x47oZIK6/\nrW4GkASkvRC+BfgY8HhvO5hZA/B14CxgGnChmU2rTHhlElf/9Ac+ULy4/IEP9P+YUfrzP/ABGDky\nf9vIkcXPHyXWU0+FYcPytw0bFmwvdMMNxY97ww352+K6GSCuv22UeEWSEOZyJK4F+CnQ2strJwFr\nc9avA67r65ip6Z7K7WbIdi8UrvdHR4f72LHBccaOLb4eVW63TLY7pnA968AB96FDg9eGDi2+3p9Y\n9+93b2gIXmtoKL4e9bhR/gZR2iCuv21cxxUJgZDdU2l+3Osk4Pc5623ACcV2NLPFwGKA5ubm+CML\nI/dxpbff3j3AbqAzljY0wJIlsHJlcBdStntm7Nhge3+6qLKF72zhN3uMYoP7Bg+GOXNgwwZob+/u\nOho6NNie210UJdYhQ2DSJNi2LegSynZRNTQE23O7qBoaYNo0eO65nsedNq37uFEeIdvbzQCzZ/e8\nGSCuv60eeStVILaahpk9Cowv8tJyd/9eZp+fAp/3IjWNTN3jTHe/NLP+FwT1jytLnTeVNY04+qfj\nqmmE7c+vtppG2NpDlLpOXH/a6CAdAAAHoElEQVTbuGolIiUkXtNw99PdfXqR5XshD9EGHJWz3gRs\nL3+kMXIP/tWYa+nS3vunC7f3tl9XF7QW/G1bW4v3vUc5Ztj+/K6u4Koi15w5ve8bNtauLpg7N3/b\n3Lm9f66rr87fdvXVxT9flMfYht3XIw5EjEKP3ZU0C9OHFddC6ZrGYOAl4GigEXgWOLavY6aqpjFp\nUtAffeWVwfqVVwbrkyb17J9esSK/3zrbn71iRf5+Ufre4zhm1H3Hjy++7/jx/T9uXH3/Ydurq8v9\nhBOC8111VbB+1VXB+gknqPYgVYmQNY1E7p4ys/PMrI2g2P0fZrY2s32imT0M4O4dwBXAWuB54H53\n/1US8faLe/eI53XrgvV164L1zs78f5F6hEFdYQfixXHMqPuadXcXnXpqsJ69E6qhof8DDOOYDTZK\ne4nUszCZpZqW1FxpuEcfBBdlUFeYgXhxHDPqvrlXWNkle+U10BiiDFwMI+qgyezVRXbJXnWIVCHS\nPLgvTqkrhEcdLFbuwmocx6zGGMKKEms1fS6RPiReCBeiFZez3SG5BjqoK45j9ieGJB/hGkWU9kpD\n22bjKLUuUm5hLkeqaUlN91TShd00DBSrpoJxlPZKQ9u6hy/ci4RADQzuq25RB4uVe1CXBopFE6W9\n0tC2nlO4h+D8uVPhu8Z2SDxU04hb1MFi5R7UFccxo54/6Ue4RhGlvdLQtmEf5yvSBz3uVdJDBeP4\nqG2lTFQIj5OKj+GlpWBci9S2kgAljaj0OM7wcrtP+nrkrESjtpWEqBAehYqP0aShYFyr1LaSENU0\nolLxMbqkC8a1TG0rZaJCeJzSUHzUl4WIlJEK4XFJQ/FRdRURSYiSRhRpKD7m1lU0G6uIVJgK4VGk\nofgY16NGRURCUE2jP9JQT0hDXUVEaoZqGnFK+nGcaairiEhdUtKoNmmoq4hI3VJNo9qkoa4iInVL\nNY1qlYa6iojUDNU0al3SdRURqUtKGiIiEpqShoiIhKakISIioSlpiIhIaEoaIiISmpKGiIiEVnPj\nNMxsJ7B1AIcYC+wqUzi1Qm3Sk9qkJ7VJT9XUJpPdfVxfO9Vc0hgoM9sYZoBLPVGb9KQ26Ult0lMt\ntom6p0REJDQlDRERCU1Jo6dVSQeQQmqTntQmPalNeqq5NlFNQ0REQtOVhoiIhKakUcDMvmhmvzaz\nX5jZd83ssKRjSgMzu8DMfmVmXWZWU3eDRGFmC8zsv83sRTO7Nul40sDMvmlmr5rZlqRjSQszO8rM\nfmJmz2f+v1madEzloqTR04+A6e7eAvwGuC7heNJiC/Ax4PGkA0mKmTUAXwfOAqYBF5rZtGSjSoV/\nABYkHUTKdABXu/sxwInA5bXy34qSRgF3f8TdOzKr64GmJONJC3d/3t3/O+k4EjYXeNHdX3L3/cC/\nAucmHFPi3P1x4LWk40gTd9/h7k9nfn8LeB6YlGxU5aGkUdpfAWuSDkJSYxLw+5z1Nmrki0DiY2ZT\ngOOBnycbSXnU5TPCzexRYHyRl5a7+/cy+ywnuMRcXcnYkhSmXepcsccj6vZD6ZWZjQC+Ayxz9zeT\njqcc6jJpuPvppV43s08D/wv4sNfRPcl9tYvQBhyVs94EbE8oFkk5MxtCkDBWu/sDScdTLuqeKmBm\nC4AvAOe4+76k45FU2QBMNbOjzawRWAg8lHBMkkJmZsA9wPPu/uWk4yknJY2evgaMBH5kZpvN7I6k\nA0oDMzvPzNqAk4D/MLO1ScdUaZkbJK4A1hIUNu93918lG1XyzOxfgKeAPzWzNjO7JOmYUuBk4C+A\nD2W+Rzab2dlJB1UOGhEuIiKh6UpDRERCU9IQEZHQlDRERCQ0JQ0REQlNSUNEREJT0hApYGadObdJ\nbs5MAxH1GIeZ2WfLH93B47/PzJ4ys3fM7PNxnUekkG65FSlgZn909xEDPMYU4AfuPj3i+xrcvTPE\nfkcCk4GPAq+7+5f6E6dIVLrSEAnBzBoyz1rZkHnWymWZ7SPM7DEze9rMfmlm2VlvbwXek7lS+aKZ\nzTOzH+Qc72tmdnHm95fN7AYz+xlwgZm9x8x+aGabzGydmb2vMB53f9XdNwAHYv/wIjnqcu4pkT4M\nM7PNmd9/6+7nAZcAe9x9jpkdAjxhZo8QzHp7nru/aWZjgfVm9hBwLcFzWWYCmNm8Ps7Z7u6nZPZ9\nDPiMu79gZicA3wA+VO4PKdIfShoiPb2d/bLPMR9oMbOPZ9ZHA1MJJjG8xcxOA7oIpkp/Vz/OeR8c\nnBX1/cC/BdMXAXBIP44nEgslDZFwDLjS3fPm3Mp0MY0DZrv7ATN7GRha5P0d5HcHF+6zN/NzEPBG\nkaQlkgqqaYiEsxZYkpnuGjN7r5kdSnDF8WomYXyQoDgN8BbBxJdZW4FpZnaImY0GPlzsJJlnLvzW\nzC7InMfMbEY8H0kkOl1piIRzNzAFeDoz7fVOgjuXVgPfN7ONwGbg1wDuvtvMnjCzLcAad7/GzO4H\nfgG8ADxT4lyLgJVm9n+BIQSPlX02dwczGw9sBEYBXWa2DJhWKw/6kfTSLbciIhKauqdERCQ0JQ0R\nEQlNSUNEREJT0hARkdCUNEREJDQlDRERCU1JQ0REQlPSEBGR0P4/PhN0JQieEAMAAAAASUVORK5C\nYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb3b3588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实，PLA算法的效率还算不错，只需要数次更新就能找到一条能将所有样本完全分类正确的分类线。所以得出结论，对于正负样本线性可分的情况，PLA能够在有限次迭代后得到正确的分类直线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. PCA优化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "刚刚我们的数据是线性可分的，可以使用PCA来得到分类直线。但是，如果数据不是线性可分，即找不到一条直线能够将所有的正负样本完全分类正确，这种情况下，似乎PCA会永远更新迭代下去，却找不到正确的分类线。所以，对于数据不可分的情况，应该要对PCA进行改进、优化。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PCA优化的基本做法很简单，就是如果迭代更新后分类错误样本比前一次少，则更新权重系数$w$；没有减少则保持当前权重系数$w$不变。也就是说，可以把条件放松，即不苛求每个点都分类正确，而是容忍有错误点，取错误点的个数最少时的权重系数$w$。通常在有限的迭代次数里，都能保证得到最佳的分类线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 导入数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集存放在'../data/'目录下，该数据集包含了100个样本，正负样本各50，特征维度为2。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = pd.read_csv('./data/data2.csv', header=None)\n",
    "# 样本输入，维度（100，2）\n",
    "X = data.iloc[:,:2].values\n",
    "# 样本输出，维度（100，）\n",
    "y = data.iloc[:,2].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据分类与可视化"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们在二维平面上绘出正负样本的分布情况。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X2UHHWd7/H3dx5CMiGEp7mKhExA\nUYSYDJPhUSFZgxBWBR8Q4caL4cgODyoBgaOsVxLdG12vXmMCSyDgenSTddEIXPQSYNFlDRDQCSSo\nBAUxCVlQkkgCJCQmme/9o6o7PT0909UzVVPV3Z/XOXWm62F+9avupL9T9a3ft8zdERERAWhIuwMi\nIpIdCgoiIpKnoCAiInkKCiIikqegICIieQoKIiKSp6AgdcPM/t7Mbo972whtuZm9LY62RJJmGqcg\n1cjMZgHXAG8FXgXuAq53961p9qsUM3PgaHd/rsS6h4CTgd2AA88CPwLmu/uuobYvUimdKUjVMbNr\ngK8D1wFjCb5U24B/N7MR/fxO0/D1sGKfcfcxwGEEge4C4F4zs3S7JfVIQUGqipkdAHwZ+Ky73+fu\nu919HXA+QWD4RLjdXDNbZmZLzOxVYFa4bElBWxeZ2Xoz22JmXzKzdWZ2RsHvLwlfTwgvAX3SzDaY\n2WYz+2JBOyea2Uoz22pmL5nZTf0Fp4G4+3Z3fwg4BzgFeH+59s3sF+GvrzGz183s42Z2kJn91Mw2\nmdkr4etxlfZH6pOCglSbU4GRwJ2FC939dWA58L6CxecCy4ADgaWF25vZscDNwEyCv9DHAoeX2fd7\ngHcA04EbzOyd4fK9wNXAoQRf5tOBKyo8rsJj2QB0A6eVa9/dTw+3mezu+7v7HQT/r79LECTHA28A\nNw22P1JfFBSk2hwKbHb3PSXWvRSuz1np7ne7e4+7v1G07XnAT9z9YXf/K3ADwTX9gXzZ3d9w9zXA\nGmAygLuvcvfH3H1PeNZyKzC18kPr5UXg4MG07+5b3P3H7r7D3V8D5sXQH6kTWb7OKlLKZuBQM2sq\nERgOC9fnvDBAO28pXO/uO8xsS5l9/6ng9Q5gfwAzezvwLaATaCH4f7WqTFvlHA48Opj2zawFmA/M\nAA4KF48xs0Z33zvEfkmN05mCVJuVwC7gI4ULzWw0cDbws4LFA/3l/xKQv85uZqOAQwbZp0XAMwR3\nAB0A/D0w6CSxmR0BTAFWDLL9awguc50Ubp+7xKTEtZSloCBVxd23ESSabzSzGWbWbGYTCG7j3Aj8\nS8SmlgEfNLNTw6Ttlxn8l+YYgttiXzezY4DLB9OImbWY2VTg/wK/BO6N2P6fgaOK+vMGsNXMDgbm\nDKY/Up8UFKTquPv/Jvhr+ZsEX5aPE1wKmh713n53/y3wWeDfCM4aXgNeJjgLqdS1wH8P27gNuKPC\n37/JzF4j+HL/NvBjYIa790Rsfy7wvfDupPPDNkYRXEp7DLiv0gOS+qXBayKAme0PbCW4RPPHtPsj\nkhadKUjdMrMPhpdsRhOcdfwaWJdur0TSpaAg9excgls/XwSOBi5wnTpLndPlIxERydOZgoiI5FXd\n4LVDDz3UJ0yYkHY3RESqyqpVqza7e2u57aouKEyYMIHu7u60uyEiUlXMbH2U7XT5SERE8hILCmb2\nDjNbXTC9amZXFW1jZrbQzJ4zs6fMrCOp/oiISHmJXT5y998B7QBm1gj8F8HTsQqdTXAr4NHASQQ1\nXk5Kqk8iIjKw4copTAf+4O7F17TOBb4f3hv+mJkdaGaHuftLlTS+e/duNm7cyM6dO+Pqb10ZOXIk\n48aNo7m5Oe2uiEjKhisoXAD8oMTyw+ld3nhjuKxXUDCzLqALYPz48X0a2bhxI2PGjGHChAnoCYaV\ncXe2bNnCxo0bOfLII9PujoikLPFEc1iB8hyCKpZ9VpdY1mc0nbsvdvdOd+9sbe17R9XOnTs55JBD\nFBAGwcw45JBDdJYlIsDw3H10NvCEu/+5xLqNwBEF8+MISg5UTAFh8PTeicRn6VKYMAEaGoKfS5eW\n+41sGY6gcCGlLx0B3ANcFN6FdDKwrdJ8gohIVixdCl1dsH49uAc/u7qqKzAkGhTCxwK+j4KHrJvZ\nZWZ2WTh7L/A88BxBnfhBP+w8bY2NjbS3tzNx4kQ+9rGPsWPHjorbuOSSS3j66acB+OpXv9pr3amn\nnhpLP0UkOV/8IhT/19+xI1heLaquIF5nZ6cXj2heu3Yt73znO1PqUWD//ffn9ddfB2DmzJlMmTKF\nz33uc7G0Nxyy8B6KVLuGhuAMoZgZ9PT0XT6czGyVu3eW264uRzQnfc3vtNNO47nnngPgW9/6FhMn\nTmTixIl8+9vfBmD79u28//3vZ/LkyUycOJE77ggepDVt2jS6u7v5whe+wBtvvEF7ezszZ84EgiAB\n8PGPf5x77703v69Zs2bx4x//mL1793LddddxwgknMGnSJG699dZ4D0pEyipxc+SAyzPJ3atqmjJl\nihd7+umn+yzrz5Il7i0t7kE8D6aWlmD5UIwePdrd3Xfv3u3nnHOO33zzzd7d3e0TJ070119/3V97\n7TU/9thj/YknnvBly5b5JZdckv/drVu3urv71KlT/Ve/+lWv9orbv/POO/2iiy5yd/ddu3b5uHHj\nfMeOHX7rrbf6P/zDP7i7+86dO33KlCn+/PPPR+5/Je+hiJSW1PdLHIBuj/AdW3dnCkld88v9Zd/Z\n2cn48eP51Kc+xcMPP8yHP/xhRo8ezf77789HPvIRVqxYwbve9S4efPBBPv/5z7NixQrGjh0beT9n\nn302P//5z9m1axfLly/n9NNPZ9SoUTzwwAN8//vfp729nZNOOoktW7bw7LPPDu2gRKQiM2fC4sXQ\n1hZcMmprC+bDE/6qUHVVUodqw4bKlkc1atQoVq9e3WuZ95Ovefvb386qVau49957uf766znzzDO5\n4YYbIu1n5MiRTJs2jfvvv5877riDCy+8ML+vG2+8kbPOOmtoByIiQzJzZnUFgWJ1d6YwnNf8Tj/9\ndO6++2527NjB9u3bueuuuzjttNN48cUXaWlp4ROf+ATXXnstTzzxRJ/fbW5uZvfu3SXbveCCC/ju\nd7/LihUr8kHgrLPOYtGiRfnf+f3vf8/27dvjPygRqWl1d6Ywb15w33DhJaSWlmB53Do6Opg1axYn\nnngiENxyevzxx3P//fdz3XXX0dDQQHNzM4sWLerzu11dXUyaNImOjg6WFmXCzzzzTC666CLOOecc\nRowYkW973bp1dHR04O60trZy9913x39QIlLT6vKW1KVLgxzChg3BGcK8edV9uhcH3ZIqEo8rrgjy\nCHv3QmNj8EfozTen3avot6TW3ZkCVP81PxHJpiuugMIT/717981nITBEUXc5BRGRpCxeXNnyLFJQ\nEBGJyd69lS3PIgUFEZGYNDZWtjyLFBRERCKIUh6nq6v07/a3PIvqMtEsIlKJXEns3K3suZLY0Pum\nlVwyOYt3H0WlM4WYmBnXXHNNfv6b3/wmc+fOjX0/KqktMvwqKY9z882wZ09Q+WjPnuoKCFCvQaF4\nbEYMYzX2228/7rzzTjZv3jzktgZSHBQeffTRRPcnIsmVx8mi+gsKc+fC1VfvCwTuwfwQ/6pvamqi\nq6uL+fPn91m3adMmPvrRj3LCCSdwwgkn8Mgjj+SXv+9976Ojo4NLL72Utra2fFD50Ic+xJQpUzju\nuONYHN7PppLaIumoiZLYUUUppTrYCTgQWAY8A6wFTilaPw3YBqwOpxvKtTmk0tk9Pe6zZwf1bGfP\nLj0/SKNHj/Zt27Z5W1ubb9261b/xjW/4nDlz3N39wgsv9BUrVri7+/r16/2YY45xd/dPf/rT/tWv\nftXd3ZcvX+6Ab9q0yd3dt2zZ4u7uO3bs8OOOO843b96c30/xft2HXlJbpbNF+pflkthREbF0dtKJ\n5gXAfe5+npmNAFpKbLPC3T+QcD8CZpD7S37BgmACmD07WD7EB9gfcMABXHTRRSxcuJBRo0bllz/4\n4IP5x2wCvPrqq7z22ms8/PDD3HXXXQDMmDGDgw46KL/NwoUL8+teeOEFnn32WQ455JB+93322Wdz\n5ZVXsmvXLu67775eJbWfeuopli1bBsC2bdt49tlnOfLII4d0rCL1JJdMrofyOIkFBTM7ADgdmAXg\n7n8F/prU/iLLBYZcQIBYAkLOVVddRUdHBxdffHF+WU9PDytXruwVKKD/0toPPfQQDz74ICtXrqSl\npYVp06axc+fOAferktoiyaqX8jhJ5hSOAjYB3zWzJ83sdjMbXWK7U8xsjZktN7PjSjVkZl1m1m1m\n3Zs2bRpar3I5hEKFOYYhOvjggzn//PP5zne+k1925plnctNNN+Xnc89deM973sMPf/hDAB544AFe\neeUVIPhr/qCDDqKlpYVnnnmGxx57LP+7KqktIklKMig0AR3AInc/HtgOfKFomyeANnefDNwIlKz1\n7O6L3b3T3TtbW1sH36NcQFiwILhk1NMT/FywINbAcM011/S6C2nhwoV0d3czadIkjj32WG655RYA\n5syZwwMPPEBHRwfLly/nsMMOY8yYMcyYMYM9e/YwadIkvvSlL3HyySfn28qV1J5Z4k+WM888k1/8\n4hecccYZvUpqH3vssXR0dDBx4kQuvfRS9uzZE8txikgNipJ4GMwEvBlYVzB/GvD/yvzOOuDQgbYZ\n6jOafc6c3knlXLI5TAoPp507d/ru3bvd3f3RRx/1yZMnD3sfcpRoFsmmJUvc29rczYKfg01uk3ai\n2d3/ZGYvmNk73P13wHTg6cJtzOzNwJ/d3c3sRIIzly1J9QkIbj1135dDyOUYYsopVGLDhg2cf/75\n9PT0MGLECG677bZh74OIZFfUkdRxSvruo88CS8M7j54HLjazywDc/RbgPOByM9sDvAFcEEa0ZBUH\ngBQCAsDRRx/Nk08+mcq+RST7BhpJXZVBwd1XA8VP+rmlYP1NwE3EwN2xlL7cq91wxGERqVwaI6lr\nYkTzyJEj2bJli77cBsHd2bJlCyNHjky7KyJSJI2R1DVRJXXcuHFs3LiRId+uWqdGjhzJuHHj0u6G\nSCT19Iz1efN65xQAWlqC5UmpiaDQ3NysEboidSCNxGua0hhJbdV2yaWzs9O7u7vT7oaIpGDChCAQ\nFGtrg3Xrhrs31cXMVrl7cY63j5rIKYhIfainEtZpUVAQkapRVyWsU6KgICJVY968INFaKOnEa71R\nUBCRqjFzZvD847a2YMxpW1swX4tJ5rTUxN1HIlI/6qWEdVp0piAiInkKCiIikqegICKZsHRpMA6h\noSH4uXRpttrL+n7jopyCiKQu7pHKaY18roUR1xrRLCKpi3ukclojn7M84lojmkWkasQ9Ujmtkc+1\nMOJaQUFEUhf3SOW0Rj7XwojrRIOCmR1oZsvM7BkzW2tmpxStNzNbaGbPmdlTZtaRZH9EBqPaE4fV\nYN48aG7uvay5efAjldMa+VwLI66TPlNYANzn7scAk4G1RevPBo4Opy5gUcL9EalILnG4fn3waO9c\n4lCBIX5xPiU3rZHPtTDiOrFEs5kdAKwBjurvuctmdivwkLv/IJz/HTDN3V/qr10lmmU4ZTlxWEv0\nPicvC4nmo4BNwHfN7Ekzu93MRhdtczjwQsH8xnBZL2bWZWbdZtatp6vJcKqFxGE10PucHUkGhSag\nA1jk7scD24EvFG1T6gSxz1mFuy92905372xtbY2/pyL9qIXEYTXQ+5wdSQaFjcBGd388nF9GECSK\ntzmiYH4c8GKCfRKpSC0kDnOyPMK3lt7nqufuiU3ACuAd4eu5wDeK1r8fWE5wxnAy8MtybU6ZMsVF\nhtOSJe5tbe5mwc8lS9LuUeWWLHFvaXEP0uXB1NKS/LFUst9aeJ+zDOj2CN/biY5oNrN24HZgBPA8\ncDHw8TAY3WJmBtwEzAB2ABe7+4BZZCWaRSqnEb4SNdGsMhcidaChIfg7vZgZ9PTU3n6lryzcfSQi\nGaERvhKVgoLIMEujRHQ1jPDVyPGMiJJ4yNKkRLNUs7gTvtWQyI2y37QS4fWELCSak6CcglSzWikR\nHbdaOY4sU05BJINqpUR03GrlOGqBgoLIMKqVEtFxq5XjqAUKCiLDKO6E77x5QWK2UEPD0BLIaSR8\nNaI5OxQURIZR3KWVH3mk7/3+PT3B8sFIq1R4LZScrhVKNItUsaYm2Lu37/LGRtizp/L2lPCtXUo0\ni9SBUgFhoOXlKOErCgoiVayxsbLl5SjhKwoKUnPiTpRecUVwmcYs+HnFFUPbLk5dXZUtL0cJX1FQ\nkJoSd6L0iitg0aJ9l2P27g3mi7/wo24Xt3e/u+9ZQWNjsHwwlPAVJZqlpsSdKI2ayI074RuVEsMS\nlRLNUpfiTpRGTeTGnfCNSolhiZuCgtSUuBOlURO5cSd8o1JiWOKWaFAws3Vm9mszW21mfa75mNk0\nM9sWrl9tZjck2R+pfXEnSqMmcitJ+MaZCFdpaoldlFKqg52AdcChA6yfBvy0kjZVOlvKibtE9OWX\nuzc2BuWcGxuD+VL7bGrqXfq5qanvvpMoEa3S1BIFWSidbWbrgE5339zP+mnAte7+gahtKtEsWRQ1\n4atnJUtaspJoduABM1tlZv3dOX2Kma0xs+VmdlypDcysy8y6zax706ZNyfVWZJCiJnzTSgwrIS1R\nJR0U3u3uHcDZwKfN7PSi9U8Abe4+GbgRuLtUI+6+2N073b2ztbU12R6LDELUhK+elSxZl2hQcPcX\nw58vA3cBJxatf9XdXw9f3ws0m9mhSfZJJCeNhK+elSyZFyXxMJgJGA2MKXj9KDCjaJs3s28A3YnA\nhtx8f5MSzRKHtBK+lWwXNyWk6xtpJ5rN7CiCswOAJuBf3X2emV0WBqNbzOwzwOXAHuAN4HPu/uhA\n7SrRLHFQ4rU0vS+1K2qiWWUupC41NAR/Bxcz6/vQmnqi96V2ZeXuI5FMUuK1NL0v0m9QMLN3mdlj\nZvaCmS02s4MK1v1yeLonkowkEr5plM6Om0pny0BnCouAucC7gN8DD5vZW8N1zQn3SyRRcZeITqt0\ndtxUOlv6zSmY2Wp3by+Y/xtgMfA/gJs9GH8w7JRTkCxKq3S2SFRRcwpNA7dhY919G4C7/4eZfRT4\nMXBwTP0UqQlplc4WidtAl4++DryzcIG7PwVMB+5MslMi1Sat0tkices3KLj7v7r7YyWWb3D3v0u2\nWyLVJa3S2Um0J/VtoMtHIhLRzTcHPxcvDi4ZNTYGASG3PCf3DOkdO4L53DOkYXDJ3LjbE9HgNZFh\nFPeIYY1Alqg0eE0kg+IuYa2S2BK3skHBzN5uZj8zs9+E85PM7H8m3zWRIsVntVV2lgvxjxjWCGSJ\nW5QzhduA64HdkL8D6YIkOyW14YwzggFQuemMM0pvFylROncuXH31vkDgHszPnZtI3yvuX0Tz5sGI\nEb2XjRgx+BHDSYxAVuK6zpUrowr8Kvz5ZMGy1VFKsCYxqXR2dZg+vXf55dw0fXrv7SKVau7pcZ89\nO1g5e3bp+YTEXUp6yRL35ube7TU3D0/J7qhtqXR2bSJi6ewoQWE58FbgiXD+PGB5lMaTmBQUqkOp\ngJCbCrW1ld6mra2owcJAkJsSDggV9S+l9uKW9f7J4EUNCmXvPgqfi7AYOBV4BfgjMNPdS9zzkDzd\nfVQdzPpfV/hPrqJSze7BL+T09Ay8oxjEXUo666Wps94/GbxY7j4yswag093PAFqBY9z9PWkFBKk9\nkROlHuYQChXmGBJSFYnh4vdgCO+JEtcyYFBw9x7gM+Hr7e7+WiWNm9k6M/u1ma02sz5/3ltgoZk9\nZ2ZPmVkqRfakMlESkdOnl/7d4uWREqW5gLBgAf88ZjYN9PDPY2bDggWJB4Z586C5qCZwc3OGEsMx\nJ+BVOlui5BS+BFwLHEFQCO9g4OAo16aAdcChA6z/W4KchQEnA4+Xa1M5hXRVkogsTjYXJ5kL2yyX\nKF3z4Tl+U9Nsh56wvR6/qWm2r/nwnBiPrnTfRozofRwjRmQkMZxQAj6tZ0hLsogxp/DH0rHEjyoX\ncMxsHcHlp839rL8VeMjdfxDO/w6Y5u4v9demcgrpSmsEbbBfJ/j7Icdpa7Nh2G/f5ZkZMVxwFpU3\nezbMn594vkWqSyae0RwGlFcAB25198VF638K/KO7PxzO/wz4vLt3F23XBXQBjB8/fsr6Uv9LZVik\nlYist/1WJIUEvFSf2MpcmNlFpaaI/Xi3Bw/jORv4tJmdXtx8id/p81/Q3Re7e6e7d7a2tkbctSQh\nrURkuvst/ifp2Um8JpGAjzFxLdUnyojmEwqm0wge0XlOlMbd/cXw58vAXcCJRZtsJMhV5IwDXozS\ntsQvSgI5rURkJfuNc0TuPR1zWdhwNfsCg7Ow4Wru6Zg7+EbjUnjpaPbs4Axh9hAT8CmOHJeMiJJ4\nKJyAscA9EbYbDYwpeP0oMKNom/fTO9H8y3LtKtGcjEoSyGklIqPsN9YRuT09vvasIHE7nyDJPZ9g\nfu1ZyQ+ci2TOnN5J5Vyyec6cyttKceS4JI+4RjT3+QVoBtZG2O4oYE04/Rb4Yrj8MuCy8LUB/wT8\nAfg1QVJaQSEFtTKSNfYRyOP3BYLcNJ/Z3jY+Q1+QxV/WQ/nyTmnkuCQvalCIcvfRT9h37twAHAv8\nyN0/P4QTlEHT3UfJqIqEagTJjEB2vOBKq9GDmVXV+1IRJa5rUpzPU/gm8H/C6WvA6WkFBElOqiNZ\nY0xsxj4C+QhnPr0TufO5mvFHVGHyNcr7nETiWqpKlKDwt+7+n+H0iLtvNLOvJ94zGVapjWStILEZ\ndyK8bHvu3PfOq7mKBXyb2Rg9fJvZXMUC7ntnlX1RRnmfc8viTFxL9Sl3fYmwOmrRsqeiXJtKYlJO\nITnDnkCuILEZdyI8cntz5vjas4IcglmQY1h71iATuWmpJIEcZ+JaMoWh5hTM7HLgCoKE8R8KVo0B\nHnH3TyQarfqhnEKNKfzrNKfEiNxUn23s3vuaevF8NYj4Pue3rfbjlT6GPKLZzMYCBxHkEb5QsOo1\nd/9LLL0cBAWFGhQhsVlvJawToQRyXRtyotndt7n7One/0INS2W8Q3IW0v5llZTynVLuIic2KEsjF\n3+olvuUrai/GRHgi0kogZ/19kUGJUubig2b2LMHDdf6ToPLp8oT7JfWggsRm5ATytGkwZcq+QNDT\nE8xPm9Zrs8jtZX2Eb1oJ5Ky/LzJoUe4++l8Eo41/7+5HAtOBRxLtldQHMzjwwN7XtufPD+YPPLDX\npY2ZM2Hx4uCav1nwc/HiYHleTw9s2warV+8LDFOmBPPbtvU6Y4jUnjts3dr7yzP35bp1a/p/GUft\nXwXvc6z7lepULhNNmLEmGJncEL4uW44iqUl3H9WgOEfk7t3r3t7e+7ai9vZg+WD7luURvpX0TyOf\n6xoxjmh+EPgQ8I/AIcDLwAnufmpyoap/SjRLWT090Ni4b37v3t4J1kplPUGbVv+y/r5IL3GOaD4X\n2AFcBdxHcHvqB4fWPZGE9PRAR9FTXTs6Bn9LUZoJ2rQSyFGktV9JXpTTCaANOCN83UJY/TSNSZeP\npF9797q/6U3BpYzJk4P5yZOD+Te9qfJLSElUDY06OCzKdmlVNVU11apExMtHTeWChpn9HcFTzw4G\n3gocDtxCkHAWyQ4zaAr/SU+dGsxPnQpr1gTLK7200V+CFoaeoIWgrcK7gjwcJBZ1u7j7F1Va+5Xh\nUS5qAKuBEcCTBct+HSXiJDHpTEEG1NPjfuWV+5KfEMwPNak60HylbUVJ0KaVQK5EWvuVQSHGRPPj\n7n6SmT3p7sebWRNBPaRJCcerkpRolrI84wnQqP3L+nFIVYkz0fyfZvb3wCgzex/wI+AnQ+2gDFFx\nMI8jwRe1zST2HUWEkcoVJUDjPo4o7UXtXyXHIRKncqcSBIHj7wiCwbLwtUU5DQl/vxF4EvhpiXWz\ngE0El6hWA5eUa2+ol4/SepRkrJKoZBlnAjQJU6f2Hm+QG48wdeq+bdKsBhpnYliJXEkAES8f9Xum\nkKtv5O497n6bu3/M3c8LX1fy58psYO0A6+9w9/Zwur2Cdiu2dCl0dQXVMd2Dn11dQ3uw+7DzBEaT\nRm0ziX1HEXWkctSRu3EfR9T2ovYv7hHIIpXoL1pQ8BwF4MdRIkyJNsYBPwPeS/9nCjdV0uZQzhRq\n5TnEiYwmTSIBGqdKRipHSYDGfRxJJIaVyJUYEcPzFJ509+OLX1fCzJYRlN4eA1zr7h8oWj8rXL8J\n+D1wtbu/UKKdLoLbYhk/fvyU9aUK4UdQU+WSPYEkZNQ2k9h3FFkfqZzW+yISQRyJZu/nddQOfAB4\n2d1XDbDZT4AJHtzJ9CDwvZIdcV/s7p3u3tna2lppV/JSfQ5xnHKXJwoNNZkatc1K9h2n3CWjQoXV\nUCsV93uYxPtSyWcXZTuRKPo7hQD2Aq8CrwF7wte5+VfLnYIQnAFsJCi1/SeCUhlLBti+EdhWrt2h\nXD6q5JGOmZVEMjXrCdDCS0e5S0bF85WI+z3M+shnEY9++ajfEc3u3tjfuojB5nrgegAzm0Zw+ajX\nIzzN7DB3fymcPYeBE9JDliuL/MUvwoYNwRnCvHlF5ZKzLupoUvdoo2IraTOtkawNDTB2LLS3w6pV\nwfyqVcGZwtixlV9Civs9zPrIZ5FKRIkcQ52AaYSJZuArwDm+72zitwRluf8DOKZcWxrRHEoimZr1\nBGjxGcFgy2HnxP0eZn3ks9Q14hrRnDUa0VwhV/JzyNJ6D6PuV5+xRBDniGapVp5SUrhaREnQVvIe\nRmmvkr5lOfEvtSvK6USWJl0+ikijYgcWdwI5zoRv1hP/UpWIq3S2VCmVN+6fx5xAjtpeVFlP/EtN\nU06h1hV/IVX6BVWrcpddcl/k0PvLtXjbcu9hJe1V0scon50+Y4kgak5BQUHqV9wJWiV8JcOUaJbK\nRU2URilhncR+4xR3glYJX6kRCgoSmDu395dY7ktu7tze202b1ru8RK78xLRpye43ToWXembPDo5h\n9uzeVU7TbE8kRQoK0jtROlDp56glrOPeb9ziLk2tUtdSQ5RTkEDURGlhIMgpLD+R1H6TEHeCVglf\nyTAlmqVyUROlWS9hLSJ9KNEN0AAPAAANSUlEQVQslXGHq67qveyqq/pewunpgY6O3ss6Ovp/XvJA\n87llStCKZIaCggRfwKecAgsXwpVXBl/wV14ZzJ9ySu+cwlveAmvWwOTJwRnC5MnB/Fve0jswREkg\nK0ErkjkKChKdGTSFg+CnTg3mp04N5puaKn8GshK0ItkTpRZGlibVPkpIT4/7lVd6rxLMV15ZulRz\n1O3SKDktIiWh0tlSsagJ37i3E5HEKdEsgaijj6MmfCvZLkriuhJRRz6nMUJapEYkHhTMrNHMnjSz\nn5ZYt5+Z3WFmz5nZ42Y2Ien+1JWoo4+jJnwr2S5K4roSUUc+pzFCWqSGDMeZwmz6f/byp4BX3P1t\nwHzg68PQn/pQyejjqAnftBLDURPXaY2QFqklURIPg52AccDPgPcSPqO5aP39wCnh6yZgM+GAuv4m\nJZorsHeve3t772Rve3v/zzaO8xnNURPSUemZxSJDQhYSzWa2DPgaMAa41t0/ULT+N8AMd98Yzv8B\nOMndNxdt1wV0AYwfP37K+vXrE+tzzYl79HEl4k40K8EtMmipJ5rN7APAy+6+aqDNSizrE6XcfbG7\nd7p7Z2tra2x9HFZpJD9zl4wKFeYYisXZx7gTzXEnwkWktCinE4OZCM4QNgLrgD8BO4AlRdvUx+Wj\nOJ/fG1XhpaPcJaPi+aT62NPjftJJvS8Z5S4lnXRS5Zdy9MxikSEj4uWjxM4U3P16dx/n7hOAC4Cf\nu/snija7B/hk+Pq8cJva+pMureRnQwOMHdu7gumqVcH82LG9L6+k1ceosp4IF6klUSLHUCdgGmGi\nGfgKcE74eiTwI+A54JfAUeXaqsozhTSTn8VnBAMlmePsY9yJ5lybA81Xup1IHSELieYkVO2I5mpI\nfqaVGBaRxKWeaJYCaSY/oyaP4+6jEr4iVUlBIWmF1+eHuzx01NG9cfcxzWMWkSFpSrsDNa+/5CcM\n3yhgCPZZ+EXtXj5BO9g+pnXMIjJkyikMl8Iv4VLzSe2zkucfx93HNI5ZRErSM5oloGSviKBEs0Ay\n5atFpKYpKNQqT6B8tYjUPAUFERHJU1CoVWawcuW+s4OGhn1nDStXKq8gIiUp0VzrlGgWEZRoFqh8\nVHHU0c8iUrMUFGpVpaOK9WxjEUEjmmtXJaOKKxn9LCI1TTmFWhd1VHGlo59FpKpoRLNUTklpkZql\nRPNQ1VvStZKkdL29NyJ1JLGgYGYjzeyXZrbGzH5rZl8usc0sM9tkZqvD6ZKk+lOReku6VpKUrrf3\nRqTOJHmmsAt4r7tPBtqBGWZ2cont7nD39nC6PcH+RJP15xUnIeqzjevxvRGpM8OSUzCzFuBh4HJ3\nf7xg+Syg090/E7WtYckp1GvSNUpSul7fG5Eql4lEs5k1AquAtwH/5O6fL1o/C/gasAn4PXC1u79Q\nop0uoAtg/PjxU9avX59Yn/OUdO2f3huRqpOJRLO773X3dmAccKKZTSza5CfABHefBDwIfK+fdha7\ne6e7d7a2tibZ5dwO9Xzh/ui9Ealpw3L3kbtvBR4CZhQt3+Luu8LZ24Apw9GfAen5wv3TeyNS8xIb\n0WxmrcBud99qZqOAM4CvF21zmLu/FM6eA6xNqj+R6fnC/dN7I1LzEsspmNkkgstBjQRnJD9096+Y\n2VeAbne/x8y+RhAM9gB/IUhEPzNQu8M2eE3PF+6f3huRqpOJRHMSNKJZRKRymUg0Z1LU0bgatSsi\ndai+gkLU0bgatSsidap+gkLU0bgatSsiday+cgpRR+Nq1K6I1BglmvsTdTSuRu2KSA1RormUqKNx\nNWpXROpU/QSFqKNxNWpXROpY/TyjOepoXI3aFZE6Vp85hajPLNaoXRGpEcop9Kf4i72/L/qo24mI\n1JD6CwoiItIvBQUREclTUBARkTwFBRERyVNQEBGRPAWFoVKJbRGpIYkFBTMbaWa/NLM1ZvZbM/ty\niW32M7M7zOw5M3vczCYk1Z9EqMS2iNSYJM8UdgHvdffJQDsww8xOLtrmU8Ar7v42YD5Fz3DONJXY\nFpEalFiZCw+GSr8ezjaHU/E35bnA3PD1MuAmMzOvhmHWheUvFizYV2ZbJbZFpIolmlMws0YzWw28\nDPy7uz9etMnhwAsA7r4H2AYcUqKdLjPrNrPuTZs2JdnlyhQGhhwFBBGpYokGBXff6+7twDjgRDOb\nWLRJqW/PPmcJ7r7Y3TvdvbO1tTWJrg6OSmyLSI0ZlruP3H0r8BAwo2jVRuAIADNrAsYCfxmOPg2Z\nSmyLSA1KLKdgZq3AbnffamajgDPom0i+B/gksBI4D/h5VeQTQCW2RaQmJVY628wmAd8DGgnOSH7o\n7l8xs68A3e5+j5mNBP4FOJ7gDOECd39+oHaHXDo7biqxLSJVIGrp7CTvPnqK4Mu+ePkNBa93Ah9L\nqg/DQiW2RaSGaESziIjkKSiIiEiegoKIiOQpKIiISJ6CgoiI5CkoiIhIXmLjFJJiZpuA9cO4y0OB\nzcO4v6TUynFA7RyLjiNbav042ty9bJ2gqgsKw83MuqMM+Mi6WjkOqJ1j0XFki44joMtHIiKSp6Ag\nIiJ5CgrlLU67AzGpleOA2jkWHUe26DhQTkFERAroTEFERPIUFEREJE9BIWRm68zs12a22sz6PLDB\nAgvN7Dkze8rMOtLoZzkRjmOamW0L1682sxtKtZM2MzvQzJaZ2TNmttbMTilaXy2fR7njqJbP4x0F\nfVxtZq+a2VVF22T+M4l4HNXymVxtZr81s9+Y2Q/C59MUrt/PzO4IP4/HzWxCpIbdXVOQV1kHHDrA\n+r8FlhM8V/pk4PG0+zzI45gG/DTtfkY4ju8Bl4SvRwAHVunnUe44quLzKOpzI/AngsFQVfeZRDiO\nzH8mwOHAH4FR4fwPgVlF21wB3BK+vgC4I0rbOlOI7lzg+x54DDjQzA5Lu1O1yMwOAE4HvgPg7n/1\n4DnfhTL/eUQ8jmo0HfiDuxdXFsj8Z1Kkv+OoFk3AqPD59i3Ai0XrzyX4owRgGTDdrPxTwBQU9nHg\nATNbZWZdJdYfDrxQML8xXJY15Y4D4BQzW2Nmy83suOHsXERHAZuA75rZk2Z2u5mNLtqmGj6PKMcB\n2f88il0A/KDE8mr4TAr1dxyQ8c/E3f8L+CawAXgJ2ObuDxRtlv883H0PsA04pFzbCgr7vNvdO4Cz\ngU+b2elF60tF2Czez1vuOJ4gOF2eDNwI3D3cHYygCegAFrn78cB24AtF21TD5xHlOKrh88gzsxHA\nOcCPSq0usSxrnwlQ9jgy/5mY2UEEZwJHAm8BRpvZJ4o3K/GrZT8PBYWQu78Y/nwZuAs4sWiTjcAR\nBfPj6Hu6lrpyx+Hur7r76+Hre4FmMzt02Ds6sI3ARnd/PJxfRvDlWrxN1j+PssdRJZ9HobOBJ9z9\nzyXWVcNnktPvcVTJZ3IG8Ed33+Tuu4E7gVOLtsl/HuElprHAX8o1rKAAmNloMxuTew2cCfymaLN7\ngIvCOyxOJjhde2mYuzqgKMdhZm/OXVc0sxMJ/g1sGe6+DsTd/wS8YGbvCBdNB54u2izzn0eU46iG\nz6PIhfR/ySXzn0mBfo+jSj6TDcDJZtYS9nU6sLZom3uAT4avzwN+7mHWeSBNsXazer0JuCv8d9AE\n/Ku732dmlwG4+y3AvQR3VzwH7AAuTqmvA4lyHOcBl5vZHuAN4IIo/1BS8FlgaXia/zxwcRV+HlD+\nOKrl88DMWoD3AZcWLKu6zyTCcWT+M3H3x81sGcGlrj3Ak8BiM/sK0O3u9xDc4PAvZvYcwRnCBVHa\nVpkLERHJ0+UjERHJU1AQEZE8BQUREclTUBARkTwFBRERyVNQkLpjZnuLKmVOGEQbB5rZFfH3Lt/+\nMWa20sx2mdm1Se1HpJhuSZW6Y2avu/v+Q2xjAkElzYkV/l6ju++NsN1/A9qADwGvuPs3B9NPkUrp\nTEGE4MvazL5hZr+y4FkAl4bL9zezn5nZExY8p+Lc8Ff+EXhreKbxDQtq8P+0oL2bzGxW+Hqdmd1g\nZg8DHzOzt5rZfWHRwhVmdkxxf9z9ZXf/FbA78YMXKaARzVKPRpnZ6vD1H939w8CnCMoynGBm+wGP\nmNkDBFUmP+zur4b1bx4zs3sICttNdPd2CB7MUmafO939PeG2PwMuc/dnzewk4GbgvXEfpMhgKChI\nPXoj92Ve4ExgkpmdF86PBY4mKCr21bDabA9BOeI3DWKfd0Bw5kFQuOxHtq+0/X6DaE8kEQoKIgED\nPuvu9/daGFwCagWmuPtuM1sHjOz76+yh9+XY4m22hz8bgK0lgpJIJiinIBK4n6AIWjOAmb09rDQ7\nFng5DAh/Q5D8BXgNGFPw++uBYy14Lu5YgqqVfbj7q8Afzexj4X7MzCYnc0gildOZgkjgdmAC8ERY\ningTwZ0/S4GfmFk3sBp4BsDdt5jZI2b2G2C5u19nZj8EngKeJaha2Z+ZwCIz+59AM/BvwJrCDczs\nzUA3cADQY8HD5Y8Ng4pIYnRLqoiI5OnykYiI5CkoiIhInoKCiIjkKSiIiEiegoKIiOQpKIiISJ6C\ngoiI5P1/87q0r0vRhmUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb3d0390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Original Data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "很明显，从图中可以看出，正类和负类样本并不是线性可分的。这时候，我们就需要对PCA进行改进，寻找合适的分类线。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 特征归一化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEWCAYAAABmE+CbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3X2YFfV99/H3dx8QVxCV0KQJshir\nUUJgXTBRG4UGg5gHE5NoYonWNBYraVi81Kq1CbQp3u1lGgKxPpDHuzfb1MSHNDGilKZWNJgKilaD\nEZOIcpu7QSpIQAjs+d5/zJzl7OHs7pw9Mzsz53xe1zXX7sw5Z+Y3I57vznzm9xtzd0RERJrSboCI\niGSDCoKIiAAqCCIiElJBEBERQAVBRERCKggiIgKoIIgAYGaLzWxl+PsEM/uNmTXHvI0zzexnca5z\ngG29YGZnD8e2pH6oIMiwCL+g/tvMjihZdpmZPZhisypy9xfdfZS799SyHjNzM/u9kvWudfe31d7C\neJW3UxqXCoIMpxagq9aVWED/dkVipv+pZDjdBFxtZkdVetHMzjCzx8xsZ/jzjJLXHjSzJWb2CLAH\neGu47G/M7MfhJZ4fmNlYM+s2s9fCdUwsWccyM3spfG2DmZ3ZTzsmhn81t5jZ6eG6i9NeM3shfN87\nzWydme0ws1+Z2c1mNiJ87aFwdU+Gn/u4mc00s60l2zk53IcdZvaMmZ1X8tq3zOwfzOyHZrbLzH5i\nZsf3d2DN7GIz22Jm283shrLXqm3n0WZ2r5ltM7NXw9/H97dtqR8qCDKc1gMPAleXv2BmxwA/BJYD\nY4EvAT80s7Elb7sYmAeMBraEyz4RLn8LcDywDvgmcAywCVhU8vnHgI7wtX8CvmtmIwdqsLuvCy8f\njQKOBh4Fvh2+3ANcCbwBOB2YBcwPP3dW+J6p4efvKNvfVuAHwGrgd4DPAt1mVnpJ6SLgr8LtPg8s\nqdRGM5sE3BoehzcTHL/SL/Bq29lEcAzbgQnA68DNAx0nqQ8qCDLcPg981szGlS1/P7DZ3f+Pux9w\n928DzwIfLHnPt9z9mfD1/eGyb7r7z919J7AK+Lm7r3H3A8B3gVOKH3b3le6+Pfz83wOHAdVc018O\n7AZuCNe3wd0fDdf3AnA7MCPiuk4DRgF/6+6/dfcfAfcSFIGiu939P8N96SYoZpV8DLjX3R9y933A\n54BC8cVq2xkeo7vcfY+77yIoRFH3S3KsJe0GSGNx96fN7F7gOoK/4IvezMG/+ou2EPzlX/RShVX+\nd8nvr1eYH1WcMbOrgMvCbTlwJMFfzYMys8uBmcBp7l4Il51IcCYzHWgj+P9pQ5T1hW14qbiuUPn+\n/r+S3/eU7kuldRVn3H23mW0vaXtV7TSzNmApMIfg7ARgtJk11xq0S7bpDEHSsAj4E/p++b1McImi\n1ATg/5bMD3lo3jAvuBa4EDja3Y8CdgIW8bNfAD4UnokU3UpwFnOCux8J/EWU9YVeBo4tC8fL9zeq\nXwHHlrS3jeCy0VDbeRXBmdO7wvcXLytF3TfJKRUEGXbu/jxwB7CgZPF9wIlm9odhmPtxYBLBZZQ4\njAYOANuAFjP7PMEZwoDM7NiwrZe4+3MV1vka8BszOwm4ouz1/wbe2s+qf0Jw+enPzazVzGYSXB77\n52i708edwAfM7N1hWPzX9P1/u9p2jiY4u9oRZjuLkIaggiBp+Wugt0+Cu28HPkDw1+l24M+BD7j7\nKzFt7wGCjOE5gksze6l8CarcLOBNwJ0ldxo9E752NfCHwC7gqwSFo9Ri4H+Hd/dcWPqCu/8WOA84\nF3gFuIWg6Dxb7Y65+zPAZwiC8l8BrwJbS95SbTu/DBwetutR4P5q2yT5ZHpAjoiIgM4QREQkpIIg\nIiKACoKIiIRUEEREBMhZx7Q3vOENPnHixLSbISKSKxs2bHjF3ctHBzhErgrCxIkTWb9+fdrNEBHJ\nFTMrHwWgIl0yEhERQAVBRERCKggiIgLkLEOoZP/+/WzdupW9e/em3ZRcGjlyJOPHj6e1tTXtpohI\nynJfELZu3cro0aOZOHEiZhqMsRruzvbt29m6dSvHHXdc2s0RkZTl/pLR3r17GTt2rIrBEJgZY8eO\n1dmViAB1UBAAFYMa6NiJxKe7GyZOhKam4Gd3d9otqk7uLxmJiGRBdzfMmwd79gTzW7YE8wBz56bX\nrmrUxRlC2pqbm+no6GDy5MlccMEF7Cn+i6jCZZddxk9/+lMAbrzxxj6vnXHGGbG0U0SSc8MNB4tB\n0Z49wfK8UEGIweGHH87GjRt5+umnGTFiBLfddlvV6/ja177GpEmTgEMLwo9//ONY2ikiyXnxxeqW\nZ1HDFYSkr/GdeeaZPP/88wB86UtfYvLkyUyePJkvf/nLAOzevZv3v//9TJ06lcmTJ3PHHcHDq2bO\nnMn69eu57rrreP311+no6GBueJ45alTwbPWPf/zj3Hfffb3buvTSS7nrrrvo6enhmmuu4dRTT2XK\nlCncfvvt8e6UiAxqwoTqlmeSu+dmmjZtmpf76U9/esiy/qxc6d7W5g4Hp7a2YHktjjjiCHd3379/\nv5933nl+yy23+Pr1633y5Mn+m9/8xnft2uWTJk3yxx9/3O+8806/7LLLej+7Y8cOd3efMWOGP/bY\nY33WV77+u+++2y+55BJ3d9+3b5+PHz/e9+zZ47fffrt/4QtfcHf3vXv3+rRp0/wXv/hF5PZXcwxF\npLKkvl/iAKz3CN+xDXWGkNQ1vuJf9NOnT2fChAl8+tOf5uGHH+b888/niCOOYNSoUXzkIx9h7dq1\nvOMd72DNmjVce+21rF27ljFjxkTezrnnnsuPfvQj9u3bx6pVqzjrrLM4/PDDWb16Nf/4j/9IR0cH\n73rXu9i+fTubN2+ubadEpCpz58KKFdDeDmbBzxUr8hMoQ4PdZZTUNb5ihlDK+3lW9YknnsiGDRu4\n7777uP7665k9ezaf//znI21n5MiRzJw5kwceeIA77riDiy66qHdbX/nKVzjnnHNq2xERqcncufkq\nAOUa6gxhOK/xnXXWWXzve99jz5497N69m3vuuYczzzyTl19+mba2Nj75yU9y9dVX8/jjjx/y2dbW\nVvbv319xvZ/4xCf45je/ydq1a3sLwDnnnMOtt97a+5nnnnuO3bt3x79TIlLXGuoMYcmSvvcJA7S1\nBcvj1tnZyaWXXso73/lOILit9JRTTuGBBx7gmmuuoampidbWVm699dZDPjtv3jymTJlCZ2cn3WWp\n9+zZs7nkkks477zzGDFiRO+6X3jhBTo7O3F3xo0bx/e+9734d0pE6pr1d2kji6ZPn+7lD8jZtGkT\nJ598cuR1dHcHmcGLLwZnBkuW5PsULw7VHkMRqWz+/CA36OmB5ubgD9Bbbkm7VWBmG9x9+mDva6gz\nBMj/NT4Ryab586H0hL+n5+B8FopCFA2VIYiIJGXFiuqWZ5EKgohIDHp6qlueRSoIIiIxaG6ubnkW\nqSCIiAwiypA3xZFNoy7PooYLlUVEqhF1WOticJzFu4yiSu0MwcyONbN/N7NNZvaMmXWl1ZZamRlX\nXXVV7/wXv/hFFi9eHPt2NCy2yPCrZsibW26BAweCkYwOHMhXMYB0LxkdAK5y95OB04DPmNmkxLda\n3u8ihn4Yhx12GHfffTevvPJKzesaiIbFFhl+9TCsdVSpFQR3/5W7Px7+vgvYBLwl0Y0uXgxXXnmw\nCLgH8zX+Nd/S0sK8efNYunTpIa9t27aNj370o5x66qmceuqpPPLII73L3/ve99LZ2cnll19Oe3t7\nb0H58Ic/zLRp03j729/OivCeNQ2LLZKOuhjWOqooQ6ImPQETgReBIyu8Ng9YD6yfMGHCIcO6Rh66\nuVBw7+oKxqTt6qo8P0RHHHGE79y509vb233Hjh1+0003+aJFi9zd/aKLLvK1a9e6u/uWLVv8pJNO\ncnf3z3zmM37jjTe6u/uqVasc8G3btrm7+/bt293dfc+ePf72t7/dX3nlld7tlG/XvfZhsTX8tUj/\nsjysdVREHP469VDZzEYBdwEL3f218tfdfQWwAoKhK2rYEBT/gl+2LJgAurqC5TU+bP7II4/kkksu\nYfny5Rx++OG9y9esWdP7aEyA1157jV27dvHwww9zzz33ADBnzhyOPvro3vcsX76897WXXnqJzZs3\nM3bs2H63fe6557JgwQL27dvH/fff32dY7Keeeoo777wTgJ07d7J582aOO+64mvZVpJEUg+NGGPIm\n1YJgZq0ExaDb3e8ehg0GX/7FYgCxFIOihQsX0tnZyac+9aneZYVCgXXr1vUpEtD/8NgPPvgga9as\nYd26dbS1tTFz5kz27t074HY1LLZIshplyJs07zIy4OvAJnf/0rBstJgZlCrNFGp0zDHHcOGFF/L1\nr3+9d9ns2bO5+eabe+eLz01497vfzXe+8x0AVq9ezauvvgoEf8UfffTRtLW18eyzz/Loo4/2flbD\nYotIktK8y+j3gYuB95jZxnB6X2JbKxaDZcuCy0SFQvBz2bJYi8JVV13V526j5cuXs379eqZMmcKk\nSZO47bbbAFi0aBGrV6+ms7OTVatW8bu/+7uMHj2aOXPmcODAAaZMmcLnPvc5TjvttN51FYfFnlvh\nT5XZs2fz0EMPcfbZZ/cZFnvSpEl0dnYyefJkLr/8cg4cOBDLfopI/Wms4a8XL4YdOw5eJioWiaOO\nqvlOo2rt27eP5uZmWlpaWLduHVdcccUhT10bLhr+WiSb4hquX8NfV7J4cVAEiplBMVOIKUOoxosv\nvsiFF15IoVBgxIgRfPWrXx32NohIdkXtIR2nxioIcOiXfwrFAOCEE07giSeeSGXbIpJ9A/WQTqog\n1MXgdnm67JU1OnYi2ZRGD+ncF4SRI0eyfft2fbENgbuzfft2Ro4cmXZTRKRMGj2kc3/JaPz48Wzd\nupVt27al3ZRcGjlyJOPHj0+7GSKRNNIz0Zcs6ZshALS1BcuTkvuC0Nraqp63Ig0gjZA1TWn0kM79\nbaci0hgmTgyKQLn2dnjhheFuTb5Eve009xmCiDSGRhqGOi0qCCKSCw01DHVKVBBEJBeWLAlC1VJJ\nh6yNRgVBRHJh7tzgecXt7UF/0vb2YL4eA+W05P4uIxFpHI0yDHVadIYgIiKACoKIiIRUEEQkdd3d\nQT+DpqbgZ3d3ttaX9e3GRRmCiKQq7h7IafVoroee1OqpLCKpirsHclo9mrPck1o9lUUkF+LugZxW\nj+Z66EmtgiAiqYq7B3JaPZrroSe1CoLIAPIeEubBkiXQ2tp3WWvr0Hsgp9WjuR56UqsgiPSjGBJu\n2RI8irsYEqooxC/OJ9um1aO5HnpSK1QW6UeWQ8J6ouOcPIXKIjWqh5AwD3Scs0MFQaQf9RAS5oGO\nc3aoIIj0ox5CwqIs99ytp+Oce+6em2natGkuMpxWrnRvb3c3C36uXJl2i6q3cqV7W5t7EI0HU1tb\n8vtSzXbr4ThnGbDeI3zHKlQWqXPquSsKlUUEUM9diU4FQaTOqeeuRKWCIDKM0hjmOQ89d9UjPCOi\nBA1ZmRQqS57FHe7mIbSNst20Qu9GgkJlkWypl2Ge41Yv+5FluQiVzewbZvZrM3s6zXaIDId6GeY5\nbvWyH/Ug7QzhW8CclNsgMizqZZjnuNXLftSDVAuCuz8E/E+abRAZLnGHu0uWBCFsqaam2sLiNMJd\n9VTOjrTPEEQaRtzDIz/yCBQKfZcVCsHyoUhruO96GDa6XqQeKpvZROBed5/cz+vzgHkAEyZMmLal\nUvok0oBaWqCn59Dlzc1w4ED161O4W79yESpH4e4r3H26u08fN25c2s0RyYxKxWCg5YNRuCuZLwgi\nUllzc3XLB6NwV9K+7fTbwDrgbWa21cw+nWZ7JN+SCETnzw8uzZgFP+fPr+19cZo3r7rlg1G4K6n3\nPq5mUk9l6U8SvV2vuKLv+orTFVcM7X1xW7nSvbm57zabm2vbZw1DXZ9QT2VpJEkEolFD27jD3agU\nAktUdRMqi0SRRCAaNbSNO9yNSiGwxE0FQepCEoFo1NA27nA3KoXAEjcVBKkLSQSiUUPbasLdOINv\nDS8tsYsSNGRlUqgsA0kiEL3iioPBbXNz5aB45Ur3lpa+4W5Ly6HbTyL41vDSEgUKlUWGR9RwV882\nlrQoVBYZJlHDXT3bWLJOBUGkRlHDXT3bWLJOBUEaTtwBa9RwV882lsyLEjRkZVKoLLVKKmCNGmjr\n2caSBhQqixxKAWv/dGzql0JlkQoUsPZPx0ZUEKShKGDtn46N9FsQzOwdZvaomb1kZivM7OiS1/5z\neJonEq+kgt00hr+Om4a/loHOEG4FFgPvAJ4DHjaz48PXWhNul0giknh+7/z5cOutBwez6+kJ5vNW\nFPRsY+k3VDazje7eUTL/B8AK4GLgFnfvHJ4mHqRQWbIoreGvRaKKGiq3DLwOG+PuOwHc/d/N7KPA\nXcAxMbVTJPfSGv5aJG4DXTL6O+Dk0gXu/hQwC7g7yUaJ5Elaw1+LxK3fguDu/+Tuj1ZY/qK7/0my\nzRLJj7SGv05ifdLYBrpkJCIR3HJL8HPFiuAyUXNzUAyKy4u6u4Ple/YE81u2HCwaQwlu416fiHoq\niwyTuHsCq2exRKWeyiIZE3dPYPUslrgNWhDM7EQz+zczezqcn2Jmf5l800RC5WexOTqrLRV3T2D1\nLJa4RTlD+CpwPbAfeu80+kSSjZJ8O/vsoGNTcTr77MrvixSILl4MV155sAi4B/OLFyfS9iG1MaIl\nS2DEiL7LRowYek/gJHoWK6RucIMNhwo8Fv58omTZxihDqcY9afjr7Js1q+/wycVp1qy+74s01HKh\n4N7VFbzY1VV5PkFxDwe9cqV7a2vf9bW2Jv9M5WrWpeGv6xMRh7+OUhBWAccDj4fzHwNWRVl53JMK\nQvZVKgbFqVR7e+X3tLeXrbC0CBSnYSgGVbUxpfXFLevtk6GLWhAGvcvIzN5KMGTFGcCrwC+Bue5e\n4f6GZOkuo+wz6/+10n9qTU2VowAzKBQqfLCp5OpmoTDwhmJSVRtTWF/cst4+GbpY7jIysyZguruf\nDYwDTnL3d6dRDKS+RA5EPcwMSpVmCgnKRQhcfhxqOC4KqWXAguDuBeDPwt93u/uuYWmVZFKUwHHW\nrMqfLV8eKRAtFoNly/jG6C6aKPCN0V2wbNmwFIUlS6C1bFzf1tYMhcAxB+4a/lqiZAifA64GjiUY\n1O4Y4Jgo16PinpQhpKeawLE8WC4PlEvXOVgg+uT5i/zmli6HQri+gt/c0uVPnr8oxr3rv30jRvTd\nlxEjMhICJxS4p/XMZ0kWMWYIv6xcR/yt8ZamwSlDSE9avWKD7TpQmhk47e2WeG/czPcELjmD6tXV\nBUuXDkvGIvkRNUPQ0BUSSVqBY5pBZy5C1pQCd8mX2IauMLNLKk0xNXKOmf3MzJ43s+viWKckI63A\nMc2gM9hGeUXw7ISsSQTuMYbUkj9ReiqfWjKdSfBYzfNq3bCZNQP/AJwLTAIuMrNJta5XqhclLE4r\ncKxmu3H3sv1+52KWN13JwaLgLG+6ku93Lq5txXEovVzU1RWcGXTVGLin3CtcMiBK0FA6AWOA71f7\nuQrrOR14oGT+euD6gT6jUDl+1YTFaQWOUbYbey/bQsE3nROEtEsJQu2lBPObzhmejnGDWrSob4Bc\nDJYXLap+XSn3CpdkEVdP5UM+AK3Apmo/V2E9HwO+VjJ/MXDzQJ9RQYhfvfROTWI/2iccLALFaSld\n3j4hQ1+O5V/UtXxxp9grXJIVtSBEucvoBxw8Z24iuLzzXXe/tpYzEzO7ADjH3S8L5y8G3ununy17\n3zxgHsCECROmbal024cMWS6C0wiS2I9gnY6XXFk1CphZro5NVVwhdT2K83kIXwT+Ppz+F3BWrcUg\ntJWgb0PReODl8je5+wp3n+7u08eNGxfDZqVUqr1Ty7/BB/njZCBJ7MeEY52l9A1tl3IlE47NYdAa\n5Vh7er3CJRuiFIT3uft/hNMj7r7VzP4uhm0/BpxgZseZ2QiCIbW/H8N6pQqp9U6tIsCMO/SOFD67\nc//JV7KQZXyZLowCX6aLhSzj/pNz9iUZ5VgXl8UZUkv+DHZNiXCU07JlT0W5HhVh3e8DngN+Dtww\n2PuVISRj2MPiKgLMuEPvqsLnRYt80zlBZmAWZAqbzhliaJuWasLiOENqyRRqzRDM7ApgPvDW8Au7\naDTwiLt/Mqki1R91TKsjpX+RFlXoZZv6c4jd+15DL5/Pg4jHuve9ed9fOUTNPZXNbAxwNEFuUNpp\nbJe7/08sraySCkKdiRBgNtoQ1IlRWNzQag6V3X2nu7/g7hd5MNz16wR3G40ys6z01ZS8ihhgVhUW\nl3+jV/iGrzp8jjH4TkRaYXHWj4sMSZShKz5oZpsJHozzH8ALBE9RExmaKgLMyGHxzJkwbdrBIlAo\nBPMzZ/Z5W1UhetZ77qYVFmf9uMiQRbnL6G+A04Dn3P04YBbwSKKtkvpmBkcd1fc69tKlwfxRR/W5\nlDF3LqxYEVzjNwt+rlgRLO9VKMDOnbBx48GiMG1aML9zZ58zhUjrg+BLbseOvl+cxS/WHTvS/4s4\navuqONaxblfyabDUmTCdBp4EmsLf/zNKYh33pLuM6kycvWx7etw7OvrePtTRESyvpX1Z7rlbTfvU\no7mhEWNP5TXAh4G/BcYCvwZOdfczkitTlSlUlgEVCtDcfHC+p6dvkDoUWQ9j02pf1o+L9BFnT+UP\nAXuAhcD9BLegfrC25onErFCAzs6+yzo7a7t1KM0wNq2wOIq0tivJi3IaAbQDZ4e/twGjo3wu7kmX\njKSinh73N74xuHQxdWowP3VqMP/GNw7tslESo39G7fgV5X1pjU6qUVFziYiXjFoGKxhm9icEg8sd\nAxwPvAW4jSBcFkmfGbSE/5RnzAjmZ8yAJ58Mlg/lUkZ/YSzUHsZCsK7Su3887AAW9X1xty+qtLYr\nw2OwigFsBEYAT5Qs+68o1SbuSWcI0q9CwX3Bgr5B54IFtf/FmkYYm1ZYXI20titDQoyh8k/c/V1m\n9oS7n2JmLQTjG01JuFYdQqGyDMhzEHRGbWMe9kVyI85Q+T/M7C+Aw83svcB3gR/U2kCpQXkRrzXM\ni7q+uLcbVYQeyFUFnUnsR5R1Rm1jNfsiEqMoBeE6YBvwX8DlwH3AXybZqKTE/czdVMTdSzTq+tLq\nnRqlB3KxLVF64yaxH3H2GK5mX0Ri1m9BKI5X5O4Fd/+qu1/g7h8Lf8/dv8rubpg3Lxjp0j34OW9e\nzopCaeAYRy/RqOuLe7tRRe2BHLU3bhL7EXWdUdsYd89ikWr0Fy5Q8hwE4K4ogUTSUy2hcr08Ozj2\nXqJJBJ1xqqYHcpSgM4n9SCIEVmgrMSKG5yE84e6nlP+eplpC5boa9jjuwDHrQWfcPZCT2I+0jo1I\nBHGEyt7P77mU6rOD4xR3eJr1oLN4mahUaaZQrWr3I85jWG07B9tuNe8TiaK/UwegB3gN2AUcCH8v\nzr8W5fQj7qmWS0ZVPToxq+J+HGLU9aXVO7X0clHxMlH5fDWq3Y+0egzH2aNZxKNfMuq3p7K7N/f3\nWh4Vhze+4QZ48cXgzGDJkgrDHmdZ1F6i7vH2dk2rd2pTE4wZAx0dsGFDML9hQ3CGMGZM9ZeNqtmP\nuI9hVFG3G/V9ItWIUjWyMqmnciju8DTrQWf5mUAtQ1q7V7e/WR5eWsNQS0TE1VM5S9RTuUquoLNm\naR3DqNvVf2OJIM6eypJHnlIInAflx6C/Y1LNMYy6zqjty3LQL/UrymlEViZdMopIQxT3L2oQG3eA\nH1XWg37JJeIa/lpySEMUV+ZVBLFxB/hRZT3ol7qmDKGelX8ZVfvlVI+Kl1mKX+DQ90u10vsHO4bV\nrjNqO6P8t9N/Y4kgaoaggiCNJ4kgVuGuZJhCZYkuaiAaZRjqJLYbpySCWIW7UidUEBpd1OGgowxD\nncR241R6aSeuoaWTWKdISlQQGllpIDrQ0M1Rh6GOe7txS2JoaQ1XLXVEGUKjixqIlhaBotIhJZLa\nbhKSCGIV7kqGKVSW6KIGonkYhlpEDqFQWaJxh4UL+y5buPDQyzaFAnR29l3W2dn/840Hmi8uUxAr\nkikqCI3MHU4/HZYvhwULgi/3BQuC+dNP75shvPnN8OSTMHVqcGYwdWow/+Y39y0KcT5fWESGVSoF\nwcwuMLNnzKxgZoOexkjKzKAl7NQ+Y0YwP2NGMN/SUv0zixXEimRTlPEt4p6Ak4G3AQ8C06N+TmMZ\nJaBQcF+wwPsMobxgQeWhlqO+L40ho0WkX+Rh+GszexC42t0jJcUKlRMSNdyN+30iMizqJlQ2s3lm\ntt7M1m/bti3t5uRLlJ7FUcPdat4XJaSuRtQezWn0fBapI4kVBDNbY2ZPV5g+VM163H2Fu0939+nj\nxo1Lqrn1J0rP4qjhbjXvixJSVyNqj+Y0ej6L1JnEhr9297OTWrcMorxncfE5xBs3Bp3JCoXgkk7W\nh1ouDalBzxcWSVqUoCGpCYXKyenpce/o6BvudnRUfh5xnM9Ujho+R6XnC4vUjCyHymZ2PvAVYByw\nA9jo7ucM9jmFylWKu2dxVHGHygqzRWqS6VDZ3e9x9/Hufpi7vzFKMci1NMLOYmZQqjRTGKg9tQbA\ncYbKcYfeItKvzN9llHtphJ2lA9F1dARnBh0dfUcrTaJ9cYfKcYfeIjIgPVM5SWmFnU1NMGZM39FI\ni8HymDEHL6tkPYzNeugtUm+iBA1ZmXIZKqcZdpYHyP0FynG2L+5QubjOgearfZ9IgyHLofJQ5TZU\nznrYmVYILCLDItOhckNJM+yMEhbH3T6FuyK5pYKQpDTDzjSGoVa4K5JrCpWTlPUevnG3T+GuSK4p\nQxgO5XfrDMfdO6V/rRf198ziuNuXxv6KSL/0TGVRuCsigEJlibvHsIjUPRWEehR3j2ERaQgqCCIi\nAqgg1CczWLfu4FlBU9PBs4V165QjiEhFCpXrmUJlEUGhslTTYziN4blFJHNUEOpRNT2G9SxiEQmp\np3I9itpjOOvDX4vIsFKGUM+i9BiupkeziOSSeipLdAqfReqaQuVaNFLIqvBZREIqCOUaKWRV+Cwi\nJVQQSpWGrMUvv+IX5o4d9fcb++gJAAAGv0lEQVQXcX/hc1dX/+FzIxwXkQalDKFcI4asCp9F6ppC\n5VooZK1Mx0UklxQqD5WeCVyZjotI3VNBKKVnAlem4yLSENRTuZSeCVyZjotIQ1CGUImeCVyZjotI\nLilDqEX5l5y+9AI6LiJ1rbEKQtSetuqRKyINqHEKQtSetuqRKyINqjEKQtSetuqRKyINLJVQ2cxu\nAj4I/Bb4OfApd98x2OdqCpWj9rRVj1wRqTOZ7qlsZrOBH7n7ATP7OwB3v3awz9V8l1HUnrbqkSsi\ndSTTdxm5+2p3PxDOPgqMH4aNRutpqx65ItKgspAh/DGwqr8XzWyema03s/Xbtm0b2hai9rRVj1wR\naWCJ9VQ2szXAmyq8dIO7/0v4nhuAA0B3f+tx9xXACgguGQ2xMdF62qpHrog0sNR6KpvZHwF/Csxy\n9z1RPhNLhhClp6165IpIHYmaIaQylpGZzQGuBWZELQYxbXjg+WrfJyJSR9LKEG4GRgP/amYbzey2\nlNohIiKhVM4Q3P330tiuiIj0Lwt3GYmISAaoIIiICKCCICIiIRWEWmiYbBGpIyoIQ6VhskWkzqgg\nDIWGyRaROpTKbae5VzqkxbJlB4fK1jDZIpJjqQ1dMRQ1D10RNw2TLSI5kOnhr+uChskWkTqjgjAU\nGiZbROqQMoSh0DDZIlKHlCHUQsNki0gOKEMYDhomW0TqiAqCiIgAKggiIhJSQRAREUAFQUREQioI\nIiICqCCIiEgoV/0QzGwbsCWBVb8BeCWB9eaRjkVAxyGg4xDI+3Fod/dxg70pVwUhKWa2PkqnjUag\nYxHQcQjoOAQa5TjokpGIiAAqCCIiElJBCKxIuwEZomMR0HEI6DgEGuI4KEMQERFAZwgiIhJSQRAR\nEUAFoZeZ3WRmz5rZU2Z2j5kdlXab0mBmF5jZM2ZWMLO6v82unJnNMbOfmdnzZnZd2u1Ji5l9w8x+\nbWZPp92WNJnZsWb272a2Kfz/oivtNiVJBeGgfwUmu/sU4Dng+pTbk5angY8AD6XdkOFmZs3APwDn\nApOAi8xsUrqtSs23gDlpNyIDDgBXufvJwGnAZ+r534QKQsjdV7v7gXD2UWB8mu1Ji7tvcvefpd2O\nlLwTeN7df+HuvwX+GfhQym1Khbs/BPxP2u1Im7v/yt0fD3/fBWwC3pJuq5KjglDZHwOr0m6EDLu3\nAC+VzG+ljv/nl+qY2UTgFOAn6bYkOS1pN2A4mdka4E0VXrrB3f8lfM8NBKeJ3cPZtuEU5Tg0qErP\nQNV92YKZjQLuAha6+2tptycpDVUQ3P3sgV43sz8CPgDM8jruoDHYcWhgW4FjS+bHAy+n1BbJCDNr\nJSgG3e5+d9rtSZIuGYXMbA5wLXCeu+9Juz2SiseAE8zsODMbAXwC+H7KbZIUmZkBXwc2ufuX0m5P\n0lQQDroZGA38q5ltNLPb0m5QGszsfDPbCpwO/NDMHki7TcMlvKngz4AHCMLD77j7M+m2Kh1m9m1g\nHfA2M9tqZp9Ou00p+X3gYuA94ffCRjN7X9qNSoqGrhAREUBnCCIiElJBEBERQAVBRERCKggiIgKo\nIIiISEgFQRqGmfWU3Dq4MRyKoNp1HGVm8+NvXe/6TzKzdWa2z8yuTmo7IpXotlNpGGb2G3cfVeM6\nJgL3uvvkKj/X7O49Ed73O0A78GHgVXf/4lDaKTIUOkOQhmZmzeGzMB4Ln4Vxebh8lJn9m5k9bmb/\nZWbFUU//Fjg+PMO4ycxmmtm9Jeu72cwuDX9/wcw+b2YPAxeY2fFmdr+ZbTCztWZ2Unl73P3X7v4Y\nsD/xnRcp01BjGUnDO9zMNoa//9Ldzwc+Dex091PN7DDgETNbTTDq6fnu/pqZvQF41My+D1xH8NyM\nDgAzmznINve6+7vD9/4b8KfuvtnM3gXcArwn7p0UGSoVBGkkrxe/yEvMBqaY2cfC+THACQQD3d1o\nZmcBBYJhsN84hG3eAb2jZZ4BfDcYHgeAw4awPpHEqCBIozPgs+7eZ8ym8LLPOGCau+83sxeAkRU+\nf4C+l17L37M7/NkE7KhQkEQyQxmCNLoHgCvCIY4xsxPN7AiCM4Vfh8XgDwiCXoBdBIMgFm0BJpnZ\nYWY2BphVaSPhGPq/NLMLwu2YmU1NZpdEhkZnCNLovgZMBB4PhzreRnCHTzfwAzNbD2wEngVw9+1m\n9kj48PlV7n6NmX0HeArYDDwxwLbmArea2V8CrQSP6Hyy9A1m9iZgPXAkUDCzhcCken4oi2SHbjsV\nERFAl4xERCSkgiAiIoAKgoiIhFQQREQEUEEQEZGQCoKIiAAqCCIiEvr/hb7RHhnNXGMAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa219470>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 均值\n",
    "u = np.mean(X, axis=0)\n",
    "# 方差\n",
    "v = np.std(X, axis=0)\n",
    "\n",
    "X = (X - u) / v\n",
    "\n",
    "# 作图\n",
    "plt.scatter(X[:50, 0], X[:50, 1], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 0], X[50:, 1], color='red', marker='x', label='Negative')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.title('Normalization data')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 直线初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# X加上偏置项\n",
    "X = np.hstack((np.ones((X.shape[0],1)), X))\n",
    "# 权重初始化\n",
    "w = np.random.randn(3,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "显示初始化直线位置"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuUFOWdN/Dvb24Ow/0m4Qgzgwoi\njoADiLrRIUK4mF1X4iXxkBBdDQZcGVz1RJeNcE4ObhKNCBJQNHHNMm9efFVcY+QiGlc0mAiIhgiK\nUSATcyJMAJEBwkz/3j+ebvoy3TPV01X9VFV/P+fUmanqmqqnS+xfV33reUpUFUREREW2G0BERP7A\ngkBERABYEIiIKIoFgYiIALAgEBFRFAsCEREBYEEgIqIoFgQiIgLAgkBERFElthuQjX79+ml1dbXt\nZhARBcrWrVsPqGr/jtYLVEGorq7Gli1bbDeDiChQRGSvk/V4yYiIiACwIBARURQLAhERAQhYhpDO\nyZMn0djYiOPHj9tuSiCVl5dj0KBBKC0ttd0UIrIs8AWhsbER3bt3R3V1NUTEdnMCRVXR1NSExsZG\nDBkyxHZziMiywF8yOn78OPr27cti0Akigr59+/LsiogAhKAgAGAxyAGPHZF7GhqA6mqgqMj8bGiw\n3aLsBP6SERGRHzQ0ALNmAc3NZn7vXjMPADNm2GtXNkJxhmBbcXExRo8ejZqaGlx77bVojv2LyMLN\nN9+M9957DwBw3333Jb12ySWXuNJOIvLO/PnxYhDT3GyWBwULggu6dOmC7du3Y8eOHSgrK8MjjzyS\n9TYef/xxjBgxAkDbgvCb3/zGlXYSkXf27ctuuR8VXEHw+hrfpZdeig8//BAA8OCDD6KmpgY1NTV4\n6KGHAABHjx7FV77yFYwaNQo1NTVYvXo1AGDChAnYsmUL7r77bhw7dgyjR4/GjOh5Zrdu3QAAX/va\n1/Diiy+e2tcNN9yAZ555Bq2trbjrrrswbtw4jBw5Eo8++qi7b4qIOlRZmd1yX1LVwExjxozRVO+9\n916bZZmsWqVaUaEKxKeKCrM8F127dlVV1ZMnT+qVV16py5cv1y1btmhNTY1+/vnneuTIER0xYoRu\n27ZNn376ab355ptP/e2hQ4dUVbWurk7feuutpO2lbv/ZZ5/VmTNnqqrqiRMndNCgQdrc3KyPPvqo\nfv/731dV1ePHj+uYMWP0o48+ctz+bI4hEaXn1eeLGwBsUQefsQV1huDVNb7YN/qxY8eisrISN910\nE15//XVMnz4dXbt2Rbdu3fDVr34VmzZtwvnnn4+NGzfiu9/9LjZt2oSePXs63s+0adPwyiuv4MSJ\nE1i7di0uu+wydOnSBRs2bMDPf/5zjB49GuPHj0dTUxN2796d25sioqzMmAGsXAlUVQEi5ufKlcEJ\nlIECu8vIq2t8sQwhkSnKbQ0bNgxbt27Fiy++iHvuuQeTJ0/Gvffe62g/5eXlmDBhAtavX4/Vq1fj\n+uuvP7Wvhx9+GFOmTMntjRBRTmbMCFYBSFVQZwj5vMZ32WWX4bnnnkNzczOOHj2KNWvW4NJLL8Un\nn3yCiooKfOMb38Cdd96Jbdu2tfnb0tJSnDx5Mu12v/71r+OJJ57Apk2bThWAKVOmYMWKFaf+5oMP\nPsDRo0fdf1NEFGoFdYawaFHyfcIAUFFhlruttrYWN9xwAy688EIA5rbSCy64AOvXr8ddd92FoqIi\nlJaWYsWKFW3+dtasWRg5ciRqa2vRkJJ6T548GTNnzsSVV16JsrKyU9ves2cPamtroaro378/nnvu\nOfffFBGFmmS6tOFHY8eO1dQH5OzcuRPnnnuu4200NJjMYN8+c2awaFGwT/HckO0xJKL05swxuUFr\nK1BcbL6ALl9uu1WAiGxV1bEdrVdQZwhA8K/xEZE/zZkDJJ7wt7bG5/1QFJwoqAyBiMgrK1dmt9yP\nWBCIiFzQ2prdcj9iQSAickFxcXbL/YgFgYioA06GvImNbOp0uR8VXKhMRJQNp8Nax4JjP95l5JS1\nMwQRGSwivxaRnSLyBxGpt9WWXIkI7rjjjlPzDzzwABYuXOj6fjgsNlH+ZTPkzfLlQEuLGcmopSVY\nxQCwe8moBcAdqnougIsA3CoiIzzfa2q/Cxf6YZx22ml49tlnceDAgZy31R4Oi02Uf2EY1topawVB\nVf+iqtuivx8BsBPAGZ7udOFC4Pbb40VA1czn+G2+pKQEs2bNwuLFi9u8tn//flx99dUYN24cxo0b\nhzfeeOPU8i9/+cuora3FLbfcgqqqqlMF5aqrrsKYMWNw3nnnYWX0njUOi01kRyiGtXbKyZCoXk8A\nqgHsA9AjzWuzAGwBsKWysrLNsK6Oh26ORFTr682YtPX16ec7qWvXrnr48GGtqqrSQ4cO6f33368L\nFixQVdXrr79eN23apKqqe/fu1eHDh6uq6q233qr33XefqqquXbtWAej+/ftVVbWpqUlVVZubm/W8\n887TAwcOnNpP6n5Vcx8Wm8NfE2Xm52GtnYLD4a+th8oi0g3AMwDmqepnqa+r6koAKwEzdEUOOwJi\n3+CXLDETANTXm+U5Pmy+R48emDlzJpYuXYouXbqcWr5x48ZTj8YEgM8++wxHjhzB66+/jjVr1gAA\npk6dit69e59aZ+nSpade+9Of/oTdu3ejb9++Gfc9bdo0zJ07FydOnMC6deuShsV+99138fTTTwMA\nDh8+jN27d2PIkCE5vVeiQhILjgthyBurBUFESmGKQYOqPpuHHZoP/1gxAFwpBjHz5s1DbW0tbrzx\nxlPLIpEINm/enFQkgMzDY7/66qvYuHEjNm/ejIqKCkyYMAHHjx9vd78cFpvIW4Uy5I3Nu4wEwE8B\n7FTVB/Oy01hmkCgxU8hRnz59cN111+GnP/3pqWWTJ0/GsmXLTs3HnpvwxS9+EU899RQAYMOGDTh4\n8CAA8y2+d+/eqKiowK5du/Dmm2+e+lsOi01EXrJ5l9E/APgmgMtFZHt0usKzvcWKwZIl5jJRJGJ+\nLlnialG44447ku42Wrp0KbZs2YKRI0dixIgReOSRRwAACxYswIYNG1BbW4u1a9di4MCB6N69O6ZO\nnYqWlhaMHDkS3/ve93DRRRed2lZsWOwZab6qTJ48Ga+99homTZqUNCz2iBEjUFtbi5qaGtxyyy1o\naWlx5X0SUfgU1vDXCxcChw7FLxPFikSvXjnfaZStEydOoLi4GCUlJdi8eTNmz57d5qlr+cLhr4n8\nya3h+jn8dToLF5oiEMsMYpmCSxlCNvbt24frrrsOkUgEZWVleOyxx/LeBiLyL6c9pN1UWAUBaPvh\nb6EYAMDQoUPx9ttvW9k3Eflfez2kvSoIoRjcLkiXvfyGx47In2z0kA58QSgvL0dTUxM/2DpBVdHU\n1ITy8nLbTSGiFDZ6SAf+ktGgQYPQ2NiI/fv3225KIJWXl2PQoEG2m0HkSCE9E33RouQMAQAqKsxy\nrwS+IJSWlrLnLVEBsBGy2mSjh3TgbzslosJQXW2KQKqqKmDPnny3Jlic3nYa+AyBiApDIQ1DbQsL\nAhEFQkENQ20JCwIRBcKiRSZUTeR1yFpoWBCIKBBmzDDPK66qMv1Jq6rMfBgDZVsCf5cRERWOQhmG\n2haeIRAREQAWBCIiimJBICLrGhpMP4OiIvOzocFf2/P7ft3CDIGIrHK7B7KtHs1h6EnNnspEZJXb\nPZBt9Wj2c09q9lQmokBwuweyrR7NYehJzYJARFa53QPZVo/mMPSkZkEgakfQQ8IgWLQIKC1NXlZa\n2vkeyLZ6NIehJzULAlEGsZBw717zKO5YSMii4D43n2xrq0dzGHpSM1QmysDPIWGY8Dh7j6EyUY7C\nEBIGAY+zf7AgEGUQhpAwCHic/YMFgSiDMISEMX7uuRum4xx4qhqYacyYMUqUT6tWqVZVqYqYn6tW\n2W5R9latUq2oUDXRuJkqKrx/L9nsNwzH2c8AbFEHn7EMlYlCjj13iaEyEQFgz11yjgWBKOTYc5ec\nYkEgyiMbwzwHoecue4T7hJOgwS8TQ2UKMrfD3SCEtk72ayv0LiRgqEzkL2EZ5tltYXkffhaIUFlE\nfiYin4rIDpvtIMqHsAzz7LawvI8wsJ0h/BeAqZbbQJQXYRnm2W1heR9hYLUgqOprAP5msw1E+eJ2\nuLtokQlhExUV5RYW2wh32VPZP2yfIRAVDLeHR37jDSASSV4WiZjlnWFruO8wDBsdFtZDZRGpBvCC\nqtZkeH0WgFkAUFlZOWZvuvSJqACVlACtrW2XFxcDLS3Zb4/hbngFIlR2QlVXqupYVR3bv39/280h\n8o10xaC95R1huEu+LwhElF5xcXbLO8Jwl2zfdvoLAJsBnCMijSJyk832ULB5EYjOmWMuzYiYn3Pm\n5Laem2bNym55RxjukvXex9lM7KlMmXjR23X27OTtxabZszu3nttWrVItLk7eZ3Fxbu+Zw1CHE9hT\nmQqJF4Go09DW7XDXKYbA5FRoQmUiJ7wIRJ2Gtm6Hu04xBCa3sSBQKHgRiDoNbd0Od51iCExuY0Gg\nUPAiEHUa2mYT7roZfHN4aXKdk6DBLxNDZWqPF4Ho7Nnx4La4OH1QvGqVaklJcrhbUtJ2/14E3xxe\nmpwAQ2Wi/HAa7vLZxmQLQ2WiPHEa7vLZxuR3LAhEOXIa7vLZxuR3LAhUcNwOWJ2Gu3y2Mfmek6DB\nLxNDZcqVVwGr00CbzzYmG8BQmagtBqyZ8diEF0NlojQYsGbGY0MZC4KInC8ib4rIn0RkpYj0Tnjt\nd/lpHpG7GLBmxmND7Z0hrACwEMD5AD4A8LqInBV9rdTjdhF5wqtg18bw127j8NfUXkHopqrrVPWQ\nqj4A4F8BrBORiwAEJ3ggSuDF83vnzAFWrIgPZtfaauaDVhT4bGPKGCqLyDsALlPVwwnLRgJ4BkAf\nVe2bnybGMVQmP7I1/DWRU26Eyj8EcG7iAlV9F8BEAM/m1jyi8LA1/DWR20oyvaCq/yfD8n0Avu1Z\ni4gCprg48xkCUZDwtlOiHNka/tqL7VFhy3iGQETOLF9ufq5cac4UiotNMYgtj2loMMubm8383r3x\notGZ4Nbt7RGxpzJRnrjdE5g9i8kp13oqi8gwEXlZRHZE50eKyH+40UiiQuJ2T2D2LCa3OckQHgNw\nD4CTwKk7jb7uZaOIkqSexQborDaR2z2B2bOY3OakIFSoaupQFby7mjKaNMl0bIpNkyalX89RILpw\nIXD77fEioGrmFy70pO2daqNDixYBZWXJy8rKOt8T2IuexQypC1xHw6ECWAvgLADbovPXAFjrZChV\ntycOf+1/EycmD58cmyZOTF7P0VDLkYhqfb15sb4+/byH3B4OetUq1dLS5O2Vlnr/TOVstsXhr8MJ\nbg1/LSJnAlgJ4BIABwF8DGCGqqaJs7zFUNn/RDK/lvhPzXEgGjsjWLIkvqy+Hli8uP2duaDQQmC/\nt486z2mo3G5BEJEiANeo6lMi0hVAkaoecbGdWWFB8D+nBaGoKH0UIAJEImn+sCjh6mYk4nkxALJs\no4Xtuc3v7aPOc+UuI1WNwAxqB1U9arMYULg4DkRjZwiJEjMFDwUiBHYxcGdITU5C5ZdE5E4RGSwi\nfWKT5y0j33ESOE6cmP5vU5c7CkQTLhf9rHs9ihDBz7rXm8tHeSgKixYBpSkDvZeW+igEdjlw5/DX\n5CRU/jjN9JGTgMLtiaGyPdkEjqnBcmqgnLjNjgLRd6Yv0GUl9QpEotuL6LKSen1n+gIX313m9pWV\nJb+XsjKfhMAeBe62nvlM3gKfqUxushU4mv0qgMTMQFFVJZ4Hnb4PWS0G7hQsroTK0Q3NTLdcVX/e\nybZ1GguCPbYCR5tBZyBCVkuBOwWLa0NXABiXMF0K81jNK3NqXZSITBWR90XkQxG5241tkjdsBY42\ng06zj9SKoP4JWb0I3F0MqSl4OiwIqnpbwvRtABcAKOvo7zoiIsUAfgJgGoARAK4XkRG5bpey5yQs\nthU4ZrNft3vZPl+7EEuLbke8KCiWFt2O52sX5rZhNyReLqqvN2cG9TkG7pZ7hZMPOAkaEicApQB2\nZvt3abZzMYD1CfP3ALinvb9hqOy+bMJiW4Gjk/263ss2EtGdU0xIuxgm1F4MM79zive9pB1ZsCA5\nQI4FywsWZL8ty73CyVtwGCo7+eD+JYDno9MLAD4C8EMnG+9gu9cAeDxh/psAlrX3NywI7quqSv4Q\njU1VVbZblh0v3kdVZbwIxKbFqNeqSh99OKZ+UOfywZ1YBGITi0EoOC0ITkLluoTZFgB7VbUx1zMT\nEbkWwBRVvTk6/00AF6rqbSnrzQIwCwAqKyvH7E132wd1WiCCUwe8eB9mmwpNuLIqiEBEAnVssqIM\nqcPIzVD5ClX93+j0hqo2isgPXWhjI4DBCfODAHySupKqrlTVsao6tn///i7slhJZ7Z3qYoDpxfuo\nHKxYjOTQdjFuR+XgAAatTo51LDNIlKde4eQPTgrCl9Msm+bCvt8CMFREhohIGcwzFp53YbuUBWu9\nU7MIMN0OvR2Fz6pYd+7tmIcleAj1EETwEOoxD0uw7tyAfUg6OdaxZW6G1BQ8ma4lAZgN4PcAjgJ4\nN2H6GMAqJ9ejOpoAXAHgAwB/BDC/o/WZIXgj72FxFgGm26F3VuHzggW6c4rJDERMprBzSidDW1uy\nCYvdDKnJV5BrqAygJ4BqAL8AUJUw9XGyYS8mFoQQcRhguh0WZ709N0NbW7IJi8PwfqkNpwXB8dAV\nInI6gPKEM4u8P7nVk57Kn34KbN4MDB8OnHlm29HMyDvacYBZaENQe8bBsabwci1UFpF/EpHdMJeK\n/hfAHpinqIXDG28AV11lCkJFBXDOOcCVVwJ33QU8/jiwaZMpGryG6q7YNetEaa5VZxUWp36ip/mE\nzzp8Tv3v7rd/B07a5/BYu75fCp6OTiEAvAOgL4C3o/NfArDSyemH25Mnl4w+/1z1zTdVn3xS9d//\nXfXqq1XPO6/tMJe9eqmOH686c6bqokWqTz+t+vvfqx475n6bws6LDKGuTnX0aNXWVjPf2mrm6+qS\nVss2Q/D1NXUn7fOiw5nfjwu1AYeXjEoc1IyTqtokIkUiUqSqv3bptlN/6NoVGD/eTIlaW81Ql++/\nnzxt3Aj8PGFcv9itKuec03YaOJCn5emIAL16JY/MuXixea1Xr6RjNmOG+Tl/PrBvn/kmv2hRfDkA\ncyZw+DCwfTswZgywdav5uX07MHq0eT16ucTR9gBTKw4dio8kunhx8l04qnb/2zptXxbH2tX9UiA5\n6Zi2EcBVAH4Ac6bwKYBxqnqJ981L5pvRTo8cAT74oG2x+OADoLk5vl737sCwYW0LxbBhbe+RLESp\nHx65fJhEIvEiEDN6tCkORU7urs7QPj8PL51N+9w81n4/LtSGm8NfdwVwDCZvmAFz91GDqja50dBs\n+KYgZBKJAI2NbQvF+++br6OJBg82uUVqsRg0qPMfYIUuEgGKi+Pzra25H0u/h7G22uf340JJnBaE\nDi8ZqepREakCMFRVnxSRCgDFHf1dQSoqMtcgKiuBL6f052tuBnbvblsonnzSnHHEdOmS/qzinHPM\nGQelF4kAtbXJy2prgW3bcj9DSHT77bl9E3b6Td3Jel60zwlb+yXvdRQyAPg2TK/iP0bnhwJ42UlA\n4fYUyn4IkYjqJ5+ovvKK6ooVqvPmqU6bpnrmmapFRcnp58CBqhMmqH7nO6qLF6u++KLqH/+o2tJi\n+13Y1dqqOmCAOUajRpn5UaPM/IAB8aA5GzbDWFthsRMcFTWQ4GKofCuACwH8NlpAdkf7JJAbREz4\nPHAg8KUvJb924gTw4YdtzypWrwYOHoyvd9ppwNlnpz+r6N07v+/HBhGgJPpPua7OzNfVAe+8Y5Z3\n5lurrTDWVljslK39Ul44yRB+q6rjReRtVb1AREoAbFPVkflpYpzvM4R8UQUOHGhbKHbtAj76CGhp\nia/bv3+8OCRmFkOGhKsTniowbx6wdGl82dy5wEMP5fYhZSOMtRUWZ8PWfqlT3AyVfwTgEICZAG4D\nMAfAe6o6342GZoMFwYGTJ01RSBds798fX6+kBDjrrPRnFf36BfN/7iAEnU7bGIT3QoHhWqgM4G4A\nN8EMdHcLgBcBPJ5b8ygn7X07Ky2Nf7CnOngwfaFYtw74+9/j6/Xu3bZIxIb2KC+Pr5evb4UJ/QjS\nzsfa4jTo9OLbrZshMENbsiVTuACg0kkIkc8p11DZ1iMgXeV2L9EFC1Rvu011927VX/1K9cc/Vq2p\nUa2uNiF2YqgtonrWWapXXGHC78svV/3Wt0wo7lWY6KQHsu0RPd0MgRnakgfgQqj8HIBaABCRZ1T1\n6jzUJ880NACzZsX7je3da+aBNL1U/cpp4Jjt9h5+2HzjXrwY2LAB2LEjfs36yBFzFvG97wHr15v1\n/vxns14sq3jySaBHj7a3yw4fDgwdam6l7QynPZCdBp1uH79stum0jQxtyaZMlQLRsYtSf7c55XKG\nEJZnB7v+3Fun20u33o03qq5bp7p0qeqtt6pOmqQ6eHDbs4qqKtXJk82ZyLJlqi+9pLpvn7M2x84I\nEreZeMaQ2sb25rN5v9nwYnhpDkNNLkKuw1+LyDZVrU393aZcQuVQDXusLgeOTrfndL2jR9N3wnv/\nfeDzz+PrVVRk7oTXrVvyftzsgez28fNqm0QucSNUHiUinwEQAF2ivyM6r6raw4V25k1lpblMlG55\noKjL4anT7WWz365dzSWd0aPb7usvf2l7q+zvfgc89VRyxT7jjPi4T7/8ZfJ2YpePOlMUsnkfsfVt\n9Bh2st9s1iNywslphF+mXC4ZZTXssV+5HZ76Keg8dswMJ/7002Z48ZkzzXDjxcVtL0EBqj17qt5z\njxm2/M03VQ8edPf4uX0Ms+Fmj2YidX7JyMltp6HgeNhjP3M7PPVT0FleDtTUmClRXR3Q1AT85Cfm\nMtSuXcBjj5nbZH/0I3P5KOb009t2wIt1wov1WHb6Ptw+hk453a/T9Yiy4PgRmn7AjmlR2VzGcLO3\nq63LE5n6Ifz975k74R04EF+/tDS5E96wYfGi0a9f++/Xz8NLZ9M+Kmiu9VT2ExaELBVy0Pm3v6Uv\nFB9+mNwJr0+f9GcVZ50FlJXZO4ZuB/1U0NzsqUxB5EXQGSR9+gAXX2ymRC0twJ495mFGu3bFC8Xa\ntcATT8TXKy42l5oSx4UCTOeVRx9N31Pa7TOEROzRTPngJGjwyxTK4a+9wN6umbUXxB46pPq736n+\n93+rzp+vOnSoOWapwXZZmeq4carf+Ibq97+veu21qjNmqDY3t91mtvwU9FNogKFyAWNv1/Q6CmJ7\n9ADGjTMTYILoQ4eAH//YPAlv1y7gP//TdHfv0QN49VVg1ar49hsazPO1Vc09zhMmmGdwDx9ubqN1\nctz9FPRTwWGGEGa2QmA/yzaI7egYHj1qLjnNn28GCYwpLTUjz8Z07Zr5+dqJnfCc7jfb9aigMVQm\nysSLIDZ1m62tbTvhxaY9e5I74Q0alL63dmUln69NrmCoTM45/ZbpZBhqL/brJi+C2HTb/Ld/M9s8\n4wzg8suTXzt2LP2T8BoazGB+MeXlZnDAdMWiZ8/OtZWoHSwIhW7hQnOdPPaBGPtw69XLvBYzYYL5\nsIoNGRGJmCEkevY019K92q+bEi8XxS4TJV4+6kxR6Mw2u3QBzj/fTKnb+vTTtoXinXeANWuSO+F9\n4QvpC0V1dfxxokRZ4r+cQua0t6vTYajd3q/bvAhi3dymCDBggJkuuyz5tcROeIm3yz7zjOnJHVNa\nmvn52n37Zv/+qKAwQyh0TkPW2BnB9u3xZaNH5z7InI1etl5cqrIZ7jY1Ze6Elxhs9+2bvhPemWea\nTngUWgyVyTmnIWsQhqGmuFgnvMSRZWO///Wv8fWKi01RSHdWcfrp/G8SAgyVyRlVYN685GXz5gEP\nPdT2DKE25ZEYtbXAtm2d67XLXrbeKykxl4/OPhv4yleSXzt8OP1ZxcaNwPHj8fV69kx+Al7s97PP\nTn6+NoWDk95rbk8ArgXwBwARAGOd/h17KrssEjFDTAOqc+ea+blzzfz48fHerq2tqgMGmOWjRpn5\nUaPM/IAByU8vszVkNLmjtVX144/Nk/CWLFGdM0d14kTVQYOSe2uLqA4Zojp1qvlvtny56ssvqzY2\n8r+fD8HnPZV3APgqgEct7Z+yIRK/c6WuzszX1Zm7X2LDSgP2howm9xQVmTuVqquBKVOSX/v8czMG\nVOpZxaZNpoNeTLdumZ+v3bVrPt8NZclqhiAirwK4U1UdBQPMEDwQu2S0dGl82dy5bS8ZZbOejSGj\nyR5V4M9/Tn8Jau/e5E54gwenzyoGD2YnPA8FIlRmQfAJp+Gu2+tR+MU64SUG2rHps8/i63XpkrkT\nXo9APa3Xl6yHyiKyEcAX0rw0X1X/J4vtzAIwCwAqA/cAZMuc9Cx2Gu5ms56TkDobHNcnuNrrhPfX\nv7YtEm+/bfpWRCLxdQcOzNwJL/GuN8qdk6DBqwnAq2Co7I26OtXRo+OBb2urma+ri6/j9lDLTkPq\nbPD5woXnxAnV995TXbNG9Qc/UL3xRtWLL1bt0yc52C4rUx0xQnX6dNW771Z94gnV3/xGtanJ9jvw\nHfg8VCYvOe1Z7Pehlp2G1E7Xo2AoKwPOPddMqQ4caHtWsXMn8MILyZ3w+vVr2wEv1gmvtDR/7yVg\nrGQIIjIdwMMA+gM4BGC7qk5p/6+YIWQlm57Fbl6ScRo+O+U0pM4mzKbwaWkBPv44fbCd2AmvpCRz\nJ7z+/UP7byUQoXK2WBCy5HbPYqfcDpUZZlMuDh1KXyh27wZOnIiv16tX+qE9zj4bOO00e+13gfVQ\nmRLYCDtjZwiJYpePvH4esJuhstuhNxWeXr2A8ePNlKi1Fdi3L31v7SefjK9XVARccw2wenV+222D\nk6DBL1MgQ2UbYWcsQAbiwXLqvBftcztU5vOFyZYjR1S3blVtaFC9917VRx6x3aKcgKGyD9gKO4uK\nzBg0iZlBLFju2TN+huD3MNbvoTeFV7duZqyu1PG7ws5J1fDLFMgzhMRvq7EpX99aE88E0s170b7E\ns4LYFDtb6KzUv820LafrERWy33JwAAAH/UlEQVQYODxDYKicD+rzsNPt9vn9/RIVGKehMgcP8Zpm\nCDvzUYhT95Fun263z+b7JaKcsCB4KfbhGLsmH4mYn0uWeP8huXBh8j5ibUl8XrHb7bP5fokoZwyV\nveT3Hr5ut4/hLlGgMUPIh9S7dfJx907it/WYfA1DbeP9ElFG7KlMDHeJCABDZdIMPYYD9AWAiPKL\nBSGMVIGLLzYDzM2da84M5s418xdfzKJARGmxIBAREQAWhHASATZvjp8VFBXFzxY2b2aOQERpMVQO\nM4bKRASGypRNj2EnPZqJKPRYEMIomx7DTno0E1FBYE/lMHLaY9jvw18TUV4xQwgzJz2Gs+nRTESB\nxJ7K5BzDZ6JQY6ici0IKWRk+E1EUC0KqQgpZGT4TUQIWhESJIWvswy/2gXnoUPi+EWcKn+vrM4fP\nhXBciAoUM4RUhRiyMnwmCjWGyrlgyJoejwtRIDFU7iw+Ezg9Hhei0GNBSMRnAqfH40JUENhTORGf\nCZwejwtRQWCGkA6fCZwejwtRIDFDyEXqhxw/9AweF6JQK6yC4LSnLXvkElEBKpyC4LSnLXvkElGB\nslIQROR+EdklIu+KyBoR6eXpDp32tGWPXCIqYFZCZRGZDOAVVW0RkR8CgKp+t6O/yylUdtrTlj1y\niShkAtNTWUSmA7hGVWd0tG7Odxk57WnLHrlEFCJBusvoXwCs9XwvTnvaskcuERUozwqCiGwUkR1p\npn9OWGc+gBYADe1sZ5aIbBGRLfv37+9cY5z2tGWPXCIqYJ71VFbVSe29LiLfAvCPACZqO9etVHUl\ngJWAuWTUqcY47WnLHrlEVMBshcpTATwIoE5VHX/tdyVDcNLTlj1yiShE/J4hLAPQHcBLIrJdRB7J\ny16d9rRlj1wiKkBWBrdT1bNt7JeIiDLzw11GRETkAywIREQEgAWBiIiiWBCIiAgAC0JuOEw2EYUI\nC0JncZhsIgoZFoTO4DDZRBRCVvohBF7ikBZLlsSHyuYw2UQUYNaHv85GzkNXuI3DZBNRAPh96Irg\n4zDZRBQyLAidwWGyiSiEmCF0BofJJqIQYoaQCw6TTUQBwAwhHzhMNhGFCAsCEREBYEEgIqIoFgQi\nIgLAgkBERFEsCEREBIAFgYiIogLVD0FE9gPY68Gm+wE44MF2g4jHwuBxMHgcjKAfhypV7d/RSoEq\nCF4RkS1OOm0UAh4Lg8fB4HEwCuU48JIREREBYEEgIqIoFgRjpe0G+AiPhcHjYPA4GAVxHJghEBER\nAJ4hEBFRFAtClIjcLyK7RORdEVkjIr1st8kGEblWRP4gIhERCf1dFalEZKqIvC8iH4rI3bbbY4uI\n/ExEPhWRHbbbYpOIDBaRX4vIzuj/F/W22+QlFoS4lwDUqOpIAB8AuMdye2zZAeCrAF6z3ZB8E5Fi\nAD8BMA3ACADXi8gIu62y5r8ATLXdCB9oAXCHqp4L4CIAt4b53wQLQpSqblDVlujsmwAG2WyPLaq6\nU1Xft90OSy4E8KGqfqSqfwfwfwH8s+U2WaGqrwH4m+122Kaqf1HVbdHfjwDYCeAMu63yDgtCev8C\nYK3tRlDenQHgTwnzjQjx//yUHRGpBnABgN/abYl3CuqZyiKyEcAX0rw0X1X/J7rOfJjTxIZ8ti2f\nnByHApXukXe8DY8gIt0APANgnqp+Zrs9XimogqCqk9p7XUS+BeAfAUzUEN+P29FxKGCNAAYnzA8C\n8ImltpBPiEgpTDFoUNVnbbfHS7xkFCUiUwF8F8CVqtpsuz1kxVsAhorIEBEpA/B1AM9bbhNZJCIC\n4KcAdqrqg7bb4zUWhLhlALoDeElEtovII7YbZIOITBeRRgAXA/iViKy33aZ8id5U8K8A1sOEh0+p\n6h/stsoOEfkFgM0AzhGRRhG5yXabLPkHAN8EcHn0c2G7iFxhu1FeYU9lIiICwDMEIiKKYkEgIiIA\nLAhERBTFgkBERABYEIiIKIoFgQqGiLQm3Dq4PToUQbbb6CUic9xv3antDxeRzSJyQkTu9Go/ROnw\ntlMqGCLyuap2y3Eb1QBeUNWaLP+uWFVbHax3OoAqAFcBOKiqD3SmnUSdwTMEKmgiUhx9FsZb0Wdh\n3BJd3k1EXhaRbSLyexGJjXr6AwBnRc8w7heRCSLyQsL2lonIDdHf94jIvSLyOoBrReQsEVknIltF\nZJOIDE9tj6p+qqpvATjp+ZsnSlFQYxlRwesiItujv3+sqtMB3ATgsKqOE5HTALwhIhtgRj2drqqf\niUg/AG+KyPMA7oZ5bsZoABCRCR3s87iqfjG67ssAvqOqu0VkPIDlAC53+00SdRYLAhWSY7EP8gST\nAYwUkWui8z0BDIUZ6O4+EbkMQARmGOwBndjnauDUaJmXAPh/ZngcAMBpndgekWdYEKjQCYDbVDVp\nzKboZZ/+AMao6kkR2QOgPM3ftyD50mvqOkejP4sAHEpTkIh8gxkCFbr1AGZHhziGiAwTka4wZwqf\nRovBl2CCXgA4AjMIYsxeACNE5DQR6QlgYrqdRMfQ/1hEro3uR0RklDdviahzeIZAhe5xANUAtkWH\nOt4Pc4dPA4BfisgWANsB7AIAVW0SkTeiD59fq6p3ichTAN4FsBvA2+3sawaAFSLyHwBKYR7R+U7i\nCiLyBQBbAPQAEBGReQBGhPmhLOQfvO2UiIgA8JIRERFFsSAQEREAFgQiIopiQSAiIgAsCEREFMWC\nQEREAFgQiIgoigWBiIgAAP8flSR38OlI73YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb2fadd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 迭代更新训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第   0次更新，分类错误的点个数：  28\n",
      "第   1次更新，分类错误的点个数：  18\n",
      "第 2次更新，分类错误的点个数：18\n",
      "第 3次更新，分类错误的点个数：18\n",
      "第 4次更新，分类错误的点个数：18\n",
      "第 5次更新，分类错误的点个数：18\n",
      "第 6次更新，分类错误的点个数：18\n",
      "第 7次更新，分类错误的点个数：18\n",
      "第 8次更新，分类错误的点个数：18\n",
      "第 9次更新，分类错误的点个数：18\n",
      "第10次更新，分类错误的点个数：18\n",
      "第11次更新，分类错误的点个数：18\n",
      "第12次更新，分类错误的点个数：18\n",
      "第13次更新，分类错误的点个数：18\n",
      "第14次更新，分类错误的点个数：18\n",
      "第15次更新，分类错误的点个数：18\n",
      "第16次更新，分类错误的点个数：18\n",
      "第17次更新，分类错误的点个数：18\n",
      "第18次更新，分类错误的点个数：18\n",
      "第19次更新，分类错误的点个数：18\n",
      "第20次更新，分类错误的点个数：18\n",
      "第21次更新，分类错误的点个数：18\n",
      "第22次更新，分类错误的点个数：18\n",
      "第23次更新，分类错误的点个数：18\n",
      "第24次更新，分类错误的点个数：18\n",
      "第25次更新，分类错误的点个数：18\n",
      "第26次更新，分类错误的点个数：18\n",
      "第27次更新，分类错误的点个数：18\n",
      "第28次更新，分类错误的点个数：18\n",
      "第29次更新，分类错误的点个数：18\n",
      "第30次更新，分类错误的点个数：18\n",
      "第31次更新，分类错误的点个数：18\n",
      "第32次更新，分类错误的点个数：18\n",
      "第33次更新，分类错误的点个数：18\n",
      "第34次更新，分类错误的点个数：18\n",
      "第35次更新，分类错误的点个数：18\n",
      "第36次更新，分类错误的点个数：18\n",
      "第37次更新，分类错误的点个数：18\n",
      "第38次更新，分类错误的点个数：18\n",
      "第39次更新，分类错误的点个数：18\n",
      "第40次更新，分类错误的点个数：18\n",
      "第41次更新，分类错误的点个数：18\n",
      "第42次更新，分类错误的点个数：18\n",
      "第43次更新，分类错误的点个数：18\n",
      "第44次更新，分类错误的点个数：18\n",
      "第45次更新，分类错误的点个数：18\n",
      "第46次更新，分类错误的点个数：18\n",
      "第47次更新，分类错误的点个数：18\n",
      "第48次更新，分类错误的点个数：18\n",
      "第49次更新，分类错误的点个数：18\n",
      "第50次更新，分类错误的点个数：18\n",
      "第51次更新，分类错误的点个数：18\n",
      "第52次更新，分类错误的点个数：18\n",
      "第53次更新，分类错误的点个数：18\n",
      "第54次更新，分类错误的点个数：18\n",
      "第55次更新，分类错误的点个数：18\n",
      "第56次更新，分类错误的点个数：18\n",
      "第57次更新，分类错误的点个数：18\n",
      "第58次更新，分类错误的点个数：18\n",
      "第59次更新，分类错误的点个数：18\n",
      "第60次更新，分类错误的点个数：18\n",
      "第61次更新，分类错误的点个数：18\n",
      "第62次更新，分类错误的点个数：18\n",
      "第63次更新，分类错误的点个数：18\n",
      "第64次更新，分类错误的点个数：18\n",
      "第65次更新，分类错误的点个数：18\n",
      "第66次更新，分类错误的点个数：18\n",
      "第67次更新，分类错误的点个数：18\n",
      "第68次更新，分类错误的点个数：18\n",
      "第69次更新，分类错误的点个数：18\n",
      "第70次更新，分类错误的点个数：18\n",
      "第71次更新，分类错误的点个数：18\n",
      "第72次更新，分类错误的点个数：18\n",
      "第73次更新，分类错误的点个数：18\n",
      "第74次更新，分类错误的点个数：18\n",
      "第75次更新，分类错误的点个数：18\n",
      "第76次更新，分类错误的点个数：18\n",
      "第77次更新，分类错误的点个数：18\n",
      "第78次更新，分类错误的点个数：18\n",
      "第79次更新，分类错误的点个数：18\n",
      "第80次更新，分类错误的点个数：18\n",
      "第81次更新，分类错误的点个数：18\n",
      "第82次更新，分类错误的点个数：18\n",
      "第83次更新，分类错误的点个数：18\n",
      "第84次更新，分类错误的点个数：18\n",
      "第85次更新，分类错误的点个数：18\n",
      "第86次更新，分类错误的点个数：18\n",
      "第87次更新，分类错误的点个数：18\n",
      "第88次更新，分类错误的点个数：18\n",
      "第89次更新，分类错误的点个数：18\n",
      "第90次更新，分类错误的点个数：18\n",
      "第91次更新，分类错误的点个数：18\n",
      "第92次更新，分类错误的点个数：18\n",
      "第93次更新，分类错误的点个数：18\n",
      "第94次更新，分类错误的点个数：18\n",
      "第95次更新，分类错误的点个数：18\n",
      "第96次更新，分类错误的点个数：18\n",
      "第97次更新，分类错误的点个数：18\n",
      "第98次更新，分类错误的点个数：18\n",
      "第99次更新，分类错误的点个数：18\n"
     ]
    }
   ],
   "source": [
    "for i in range(100):\n",
    "    s = np.dot(X, w)\n",
    "    y_pred = np.ones_like(y)\n",
    "    loc_n = np.where(s < 0)[0]\n",
    "    y_pred[loc_n] = -1\n",
    "    num_fault = len(np.where(y != y_pred)[0])\n",
    "    \n",
    "    if num_fault == 0:\n",
    "        break\n",
    "    else:\n",
    "        r = np.random.choice(num_fault)        # 随机选择一个错误分类点\n",
    "        t = np.where(y != y_pred)[0][r]\n",
    "        w2 = w + y[t] * X[t, :].reshape((3,1))\n",
    "        \n",
    "        s = np.dot(X, w2)\n",
    "        y_pred = np.ones_like(y)\n",
    "        loc_n = np.where(s < 0)[0]\n",
    "        y_pred[loc_n] = -1\n",
    "        num_fault2 = len(np.where(y != y_pred)[0])\n",
    "        if num_fault2 <num_fault:\n",
    "            w = w2        # 犯的错误点更少，则更新w，否则w不变\n",
    "            print('第%4d次更新，分类错误的点个数：%4d' % (i, num_fault2))\n",
    "        else:\n",
    "            print('第%2d次更新，分类错误的点个数：%2d' % (i, num_fault))        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代完毕后，得到更新后的权重系数$w$，绘制此时的分类直线是什么样子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYFdWZx/Hv283SgohIcIlAg3FL\nSwAB4zJBSHAQHcddooMxboOjRFDRKCYKzxhIVBKUEDVEjVFJgnGJiQmRaGJERWURVxxBWSQuQZRF\nEAT6nT/Obehuuum63VVdd/l9nqee7rpdt+pUife9Vb86p8zdERERKUm7ASIikhtUEEREBFBBEBGR\nDBUEEREBVBBERCRDBUFERAAVBBERyVBBEBERQAVBREQyWqTdgGx84Qtf8G7duqXdDBGRvDJv3ryP\n3L1TQ8vlVUHo1q0bc+fOTbsZIiJ5xcyWRVlOl4xERARQQRARkQwVBBERAfIsQ6jL5s2bWbFiBRs3\nbky7KXmprKyMzp0707Jly7SbIiIpy/uCsGLFCtq1a0e3bt0ws7Sbk1fcnVWrVrFixQq6d++ednNE\nJGV5f8lo48aNdOzYUcWgEcyMjh076uxKRIACKAiAikET6NiJxGfaNOjWDUpKws9p09JuUXby/pKR\niEgumDYNhg+HDRvC/LJlYR5g2LD02pWNgjhDSFtpaSm9e/emR48enHHGGWyo+heRhQsvvJA33ngD\ngAkTJtT421FHHRVLO0UkOd/73vZiUGXDhvB6vlBBiMEuu+zCggULeO2112jVqhV33HFH1uu48847\nqaioAHYsCM8991ws7RSR5Cxfnt3ruajoCkLS1/j69+/P4sWLAfjJT35Cjx496NGjB7fccgsA69ev\n5z/+4z/o1asXPXr0YPr06QAMHDiQuXPncs011/DZZ5/Ru3dvhmXOM3fddVcAvvnNb/LnP/9527bO\nPfdcHnroIbZu3cpVV13FYYcdRs+ePfn5z38e706JSIO6ds3u9Zzk7nkz9e3b12t74403dnitPvff\n796mjTtsn9q0Ca83Rdu2bd3dffPmzX7iiSf6bbfd5nPnzvUePXr4p59+6uvWrfOKigqfP3++P/jg\ng37hhRdue+/q1avd3X3AgAE+Z86cGuurvf6HH37YzznnHHd337Rpk3fu3Nk3bNjgP//5z/2GG25w\nd/eNGzd63759/Z133onc/myOoYjULanPlzgAcz3CZ2xRnSEkdY2v6ht9v3796Nq1KxdccAHPPPMM\np5xyCm3btmXXXXfl1FNPZdasWXzlK1/hiSee4Oqrr2bWrFm0b98+8naOO+44/va3v7Fp0yZmzJjB\n0UcfzS677MLMmTO599576d27N4cffjirVq1i0aJFTdspEcnKsGEwdSqUl4NZ+Dl1av4EylBkdxkl\ndY2vKkOoLhTlHR144IHMmzePP//5z4wZM4bBgwdz/fXXR9pOWVkZAwcO5PHHH2f69OmcddZZ27b1\n05/+lGOPPbZpOyIiTTJsWH4VgNqK6gyhOa/xHX300fz+979nw4YNrF+/nkceeYT+/fvz3nvv0aZN\nG84++2yuvPJK5s+fv8N7W7ZsyebNm+tc75lnnskvf/lLZs2ata0AHHvssdx+++3b3vPWW2+xfv36\n+HdKRApaUZ0hjB9f8z5hgDZtwutx69OnD+eeey5f/epXgXBb6aGHHsrjjz/OVVddRUlJCS1btuT2\n22/f4b3Dhw+nZ8+e9OnTh2m1Uu/BgwdzzjnncOKJJ9KqVatt6166dCl9+vTB3enUqRO///3v498p\nESloVt+ljVzUr18/r/2AnIULF/LlL3858jqmTQuZwfLl4cxg/Pj8PsWLQ7bHUETqdsklITfYuhVK\nS8MX0NtuS7tVYGbz3L1fQ8sV1RkC5P81PhHJTZdcAtVP+Ldu3T6fC0UhiqLKEEREkjJ1anav5yIV\nBBGRGGzdmt3ruUgFQUQkBqWl2b2ei1QQREQaEGXIm6qRTaO+nouKLlQWEclG1GGtq4LjXLzLKKrU\nzhDMrIuZ/d3MFprZ62Y2Kq22NJWZMXr06G3zEydOZNy4cbFvR8NiizS/bIa8ue022LIljGS0ZUt+\nFQNI95LRFmC0u38ZOAIYYWYViW+1dr+LGPphtG7dmocffpiPPvqoyevaGQ2LLdL8CmFY66hSKwju\n/r67z8/8vg5YCOyb6EbHjYPLL99eBNzDfBO/zbdo0YLhw4czadKkHf62cuVKTjvtNA477DAOO+ww\nnn322W2v//u//zt9+vThoosuory8fFtBOfnkk+nbty+HHHIIUzP3rGlYbJF0FMSw1lFFGRI16Qno\nBiwHdqvjb8OBucDcrl277jCsa+Shmysr3UeNCmPSjhpV93wjtW3b1tesWePl5eW+evVqv/nmm33s\n2LHu7n7WWWf5rFmz3N192bJlfvDBB7u7+4gRI3zChAnu7j5jxgwHfOXKle7uvmrVKnd337Bhgx9y\nyCH+0UcfbdtO7e26N31YbA1/LVK/XB7WOioiDn+deqhsZrsCDwGXufva2n9396nAVAhDVzRhQ1D1\nDf7WW8MEMHIk/OAH4e9NsNtuu3HOOecwefJkdtlll22vP/HEE9sejQmwdu1a1q1bxzPPPMMjjzwC\nwJAhQ+jQocO2ZSZPnrztb++++y6LFi2iY8eO9W77uOOOY+TIkWzatIm//OUvNYbFfuWVV3jwwQcB\nWLNmDYsWLaJ79+5N2leRYlIVHBfDkDepFgQza0koBtPc/eFm2GAoClXFAOD66+HNN6FdO9hnn/Cz\nkcXhsssuo0+fPpx33nnbXqusrGT27Nk1igTUPzz2U089xRNPPMHs2bNp06YNAwcOZOPGjTvdrobF\nFklWsQx5k+ZdRgbcBSx09580y0arMoPq/vd/Yd99YeNGeOutUBxWr25U2LzHHnswdOhQ7rrrrm2v\nDR48mClTpmybr3puwte+9jUeeOABAGbOnMknn3wChG/xHTp0oE2bNrz55ps8//zz296rYbFFJElp\n3mX0b8C3gG+Y2YLMdHxiW6sqBrfeCqNGQWVl+Dl5Mtx4I/ToER5xtGULLF4Mb7wBq1ZlXRhGjx5d\n426jyZMnM3fuXHr27ElFRQV33HEHAGPHjmXmzJn06dOHGTNmsM8++9CuXTuGDBnCli1b6NmzJ9dd\ndx1HHHHEtnVVDYs9rI6vKoMHD+bpp5/mmGOOqTEsdkVFBX369KFHjx5cdNFFbNmypTFHT0SKQHEN\nfz1uXPj2P2lSuCxUVSR23337nUbu8PHH8P774ayhdWvYe2/o2DF0U4zJpk2bKC0tpUWLFsyePZuL\nL754h6euNRcNfy2Sm+Iarl/DX9dl3LjwgV+VEVRlCtUzA7Pw4b/HHqF4vP9+6Jr43nuw117QqVMs\ng5MsX76coUOHUllZSatWrfjFL37R5HWKSOGI2kM6TsVVEGDHwLi+ANkMOnQIZw/r1oXCsGIFfPAB\n7LlnmFo0/vAdcMABvPTSS41+v4gUtp31kFZB2Al3x5p422i9zGC33cL06aehMLz33vbCsNde0LJl\nMttuBvl0yVCkmKTRQzrvRzstKytj1apVzfPBtuuucMABUFERzhw++ABeeSWcy23alPz2Y+burFq1\nirKysrSbIiK1pNFDOu/PEDp37syKFStYuXJl82+8VStYuxYWLgxT27bQvn1enTGUlZXRuXPntJsh\nEkkxPRN9/PiaGQJAmzbh9aTkfUFo2bJl+j1v330XfvzjMO7txo1w6qkwZgz07Ztuu0QKSBoha5rS\n6CGd97ed5pSVK0M/h5/+NJw5HHts+K/Zv3/aLRPJe926hSJQW3k5LF3a3K3JL1FvO837DCGndOoU\nxkVavhwmTID58+Hoo0NBmDEjlqG2RYpVMQ1DnRYVhCS0bx8uGS1dGnpCL1sGxx8PffrA736XX0/d\nFskRRTUMdUpUEJLUpg1cemkYCuPuu8PFz6FD4ZBD4J57oJ5xiURkR+PHh/+lqks6ZC02KgjNoVUr\nOO+8MD7S9OlQVhbm998fpkyBzz5Lu4UiOW/YsHDfRnl56B5UXh7mCzFQTotC5TS4h0xh/Hh47rnQ\nwe3yy+GSS0IHOBGRGClUzmVmIVN45hn4xz/g0END5tC1K1x3HST8bGYRkbqoIKTJLNyF9Je/wJw5\nMGhQuEupvDycMaxYkXYLRaSIqCDkin794KGH4PXX4bTTQl+G/fYLPW8WL067dSKJmjYt9DMoKQk/\np03LrfXl+nZjE+XBy7ky9e3bt2lPms4n77zjfvHF7q1bu5eUuJ91lvsrr6TdKpHYxf0Q+7jXl+vb\njQKY6xE+YxUq57r33w/PbLj99jDa6n/+J1x7LVR7kppIPou7B3JaPZpzuSe1QuVCsc8+cNNN4V/a\nuHEhiD7yyJA3PPmkej9L3ou7B3JaPZoLoSe1CkK+2GMPGDs2FIaJE8PoqsccE84UHn00PCNaJA/F\n3QM5rR7NhdCTWgUh37RrB6NHwzvvwB13hAH1Tj4ZevWCX/8atmxJu4UFJe9DwjwwfvyOI8a3bNn4\nHshp9WguhJ7UKgj5qqwMLroI3noL7rsvnCEMGwYHHRS6b+bhA3tyTdVwy8uWhStzVcMtqyjEL+qT\nbaNIq0dzIfSkVqhcKCor4Q9/CF9H5s6FL34xnEkMHx6e9CZZy+WQsJDoOCdPoXKxKSkJl45efBH+\n+lc48MBQELp1gxtugE8+SbuFeacQQsJ8oOOcO1QQCo1ZCJv//nd49tlwR9L114dk6+qr4cMP025h\n3iiEkDAf6DjnDhWEQnbUUfDHP8KCBXDCCeHupG7d4DvfqfscXWoohJCwSi733C2k45z3ovRey5Wp\nqHoqJ+Gtt9wvuMC9ZUv3Fi3cv/1t94UL025VTrv/fvfycnez8DMXep1mKx967hbCcc5lqKey1Ovd\nd+HHPw63QGzcCKeeGkZb7ds37ZZJAtRzVxQqS/26dIFbbgn/t157bQih+/WDIUNg1qy0WycxU89d\niUoFoZh16hSG216+HCZMgPnzw3Dc/fuHB/jk0dmj1E89dyUqFQSB9u3DJaOlS2Hy5HDmcPzx4RLS\ngw/C1q1pt7BgpDHMcz703FWP8BwRJWjIlUmhcjPZtMn9rrvcDzggJIEHHeT+y1+6f/552i3La2kO\n85xWaBtlu7k8bHShQKGyNNnWreGhPRMmwMsvh3P9q66CCy6AXXZJu3V5p1CGeY5boexHLsuLUNnM\n7jazf5nZa2m2Q+pRWgpDh8JLL8Gf/gSdO8Oll4b/g2+8EdauTbuFeaVQhnmOW6HsRyFIO0O4BxiS\nchukIWYhU3jmGXjqKejdG665JpwxXHcdfPRR2i3MC4UyzHPcCmU/CkGqBcHdnwY+TrMNkgUzGDAA\nHn8c5swJD+n5wQ/Cuf0VV8A//5l2C3Na3OHu+PEhhK2upKRpYXEa4a56KueOtM8QJF/16xfyhddf\nh9NOC3cnde8eRlddvDjt1uWkuIdHfvbZHZ+LVFkZXm+MtIb7LoRhowtF6qGymXUDHnP3HvX8fTgw\nHKBr1659l2kMnty0ZAncfDPcfTds3gzf/Ga4lfUrX0m7ZQWrRYu67wguLW3cc5IU7hauvAiVo3D3\nqe7ez937derUKe3mSH26d4fbbguFYfToMKhez55w0knwwgtpt64g1dc9pLHdRhTuSs4XBMkz++wD\nN90UvmqOGxeGwjjiiJA3PPmkej/HqLQ0u9cbonBX0r7t9DfAbOAgM1thZhek2R6J0R57wNixoTBM\nnAgLF4bnNBxxRHiyW+2L3zFIIhC95JJwacYs/LzkkqYtF6fhw7N7vSEKdyX13sfZTOqpnMc++8z9\njjvcu3cPXVF79HCfNs198+ZYVp9Eb9eLL665vqrp4osbt1zc7r/fvbS05jZLS5u2zxqGujChnsqS\nk7Zsgd/+Fn74Q3jjDdhvv/Akt29/G1q3bvRqkwhEo4a2cYe7USkElqgKJlSWAtOiBZx9Nrz6Kjzy\nSLi0dNFFoTBMmgTr1zdqtUkEolFD27jD3agUAkvcVBAkHSUlcPLJ8OKL4XkMBx4YOreVl8MNN8An\nn2S1uiQC0aihbdzhblQKgSVuKgiSLrMQNv/976FH1ZFHwvXXh0+1q6+GDz+MtJokAtGooW024W6c\nwbeGl5bYRQkacmVSqFwkFixwP/NM95IS97Iy9xEj3JcubfBtSQSiF1+8PbgtLa07KL7//vCI6urh\nbosWO24/ieBbw0tLFChUlry3aFEYVfXee8Pn2LBhYVC9gw9Ou2U1RA139WxjSYtCZcl/BxwAd94J\nb78NI0bAAw9ARQWcfnp43GeOiBru6tnGkutUECT3dekCt9wSvuaOGRNC6L594bjjQk/olEUNd/Vs\nY8l1KgiSPzp1Conp8uXhKW7z5sHRR0P//jBjRuRhMeIOWKOGu3q2seS8KEFDrkwKlaWG9evdJ092\n79IlJKWHHur+u9+5b9lS71uSClijBtp6trGkAYXKUjQ+/xzuvx9+9KMQRB90UAifhw2Dli1rLKqA\ntX46NoVLobIUj1at4PzzwwB606dDWRmcdx7svz9MmQKffbZtUQWs9dOxkXoLgpl9xcyeN7N3zWyq\nmXWo9rcXm6d5IlkoLYWhQ+Gll+BPf4LOneHSS8NX3xtvhLVrFbDuhI6N7OwM4XZgHPAV4C3gGTP7\nUuZvLet7k0jqzOD44+GZZ+Cpp6B373AJqbycP/a+ji67fFRj8TiC3TSGv46bhr+WesMFYEGt+a8D\ni4AjgPlRAoq4J4XK0mhz5rifeqo7+ObWbfwX7S73fVkRS7Cb1vDXSdDw14WJpobKZvYycLS7r6n2\nWk/gIWAPd++YbKnakUJlabI33gjh869/He6tPPdc+O53Q97QSGkNfy0SVRyh8o3Al6u/4O6vAIOA\nh5vWPJGUVFSEoTAWLYILLwy/H3RQuCPp1Vcbtcq0hr8WiVu9BcHdf+3uz9fx+nJ3/+9kmyWSsO7d\n4bbbYMkSGD06PNazZ0846SR44YWsVpXW8NcicdNtp1Lc9tkHbrop3IA/blwYCuOII2DQIHjyyUi9\nn9Ma/jqJ9UmRixI05MqkUFkSt3at+8SJ7nvvHZLhww93f/RR961bd/q2qMNkx9kTWD2LJSrUU1mk\nCTZuhF/9KvRfWLIEevQIA+sNHRpS5EaIuyewehZLVLH1VDazA83sSTN7LTPf08y+H0cjRXJWWVl4\n1vNbb8F990FlZQieDz4YfvEL2LQp61XG3RNYPYslblEyhF8AY4DNsO1OozOTbJRIDbXPYpvzrLZF\nCzj77HAH0iOPQIcOIRzYbz+YNAnWr4+8qrh7AqtnscQtSkFo4+61h6rQ3dVSr2OOCT12q6Zjjql7\nuUiB6LhxcPnl24uAe5gfNy6RttfbxhYldLvsZKaNehFmzoQDD4QrrgjXZ264AT75pMF1jR8fhl2q\nrlWrxvcETqJnsULqItdQyADMAL5EpncycDowI0pAEfekUDn3DRpUd6/dQYNqLhcpEK2sdB81Kvxx\n1Ki65xPUYBuffdb9hBPCH9q1c7/6avcPPtjp+lq2rLm+li2Tf6ZyNutSSF2YiBgqRykI+wFPABuA\nfwLPAOVRVh73pIKQ++oqBlVTdeXldS9TXl5rhdWLQNXUDMUgqzYuWOD+zW+GT+WyMvcRI9yXLm38\n+lKS6+2TxotaEHZ6l5GZlQCnu/sDZtYWKHH3dYmdrjRAdxnlPrP6/1b9n1pJSd1RgFnIb3d4Y0m1\nq5uVlTvfUEyyaiOEAPqmm0LvZ/cQQl9zTQiiG7O+Zpbr7ZPGi+UuI3evBL6T+X19msVACkvkQNQz\nmUF11TOFBGUd2h54INx5J7z9NowYAQ88EIbKOOMMmD8/mRC49nFownFRSC1RQuW/mtmVZtbFzPao\nmhJvmeScKIHjoEF1v7f265EC0apicOut3N1uFCVUcne7UXDrrc1SFMaP3+GBa7RsGSG07dIFbrkl\ndBIYMyaE0H378lz74zim9awaizYpBI45cNfw1xIlQ1hSx/ROlOtRcU/KENKTTeBYO1iuHShXX2dD\ngejLp4z1KS1GOVRm1lfpU1qM8pdPGRvj3tXfvlatau5Lq1aNCFlXr3afMMG9Uyd38Bdaf82HMMPL\nu1Y2PrBNKHDX8NeFibhC5VyaVBDSk1bgGLZbWWu7lc0SdMa+z+vXu0+e7N65c1jRoYe6/+537lu2\nNG59KQbukl+iFoQGh64ws3PqObO4N84zlSgUKqcnrcAxzaAzsW1//jncf394LsOiRWH47WuuCSF0\n7WtUDfF0AnfJL7ENXQEcVm3qT3is5olNal2GmQ0xs/8zs8Vmdk0c65RkpBU4phl0hm3Urgje9G23\nagXnnw8LF8L06WGYjPPOCw/p+dnP4LPPoq3HEwjca7+3KeuS/BPlNKL6BLQH/pDt++pYTynwNqGf\nQyvgZaBiZ+/RJaNkRLlunFanpWy2G/f175dPGeuTS2rmF5NLEsgvKivdH3vM/cgjww7uuaf7j37k\nvmbNzt8Td4YwdmzN91atc+zYxuyV5BCSyhCAlsDCbN9Xx3qOBB6vNj8GGLOz96ggxC/ND9xs2tjs\nBauy0hceGz5gJxGKwiTC/MJjE7pOX1np/tRT7oMHhx3YfXf373/ffeXKupeP8wM85V7hkqzYCgLw\nR+APmekx4B3gxigrb2C9pwN3Vpv/FjBlZ+9RQYhfofROTWI/yrtuLwJV0yRGeXnXZvhwnDPH/dRT\nt1e2yy93X7Fix+Vqf1A35YNbIXXBiloQooTKA6rNbgGWufuKxl+k2rbeM4Bj3f3CzPy3gK+6+6W1\nlhsODAfo2rVr32V1DQAvjVYovVOT2I+wTserRW1GJWbWfMfmjTdC+PzrX4cGnXsufPe7IW9Igiuk\nLkRxhsrHu/s/MtOz7r7CzG6MoY0rgC7V5jsD79VeyN2nuns/d+/XqVOnGDYr1aXaO7X2J3gDX052\nJon96NrFmUTN0HYSl9O1SzMGrRUVYSiMRYvgwgvD7wcdFO5IevXV6OuJcqw9vV7hkiMaOoUgM8pp\nrddeiXL60cB6WxAuP3Vne6h8yM7eo0tG8UtthMssrn/HnSFEykLSyBCieO899yuvdG/bNuzkiSe6\nP//8zt8T5VgrQyhoNDVDAC4GXgXWA69Um5YA90dZeYMbh+OBtwh3G32voeVVEJLR7GFxFh8+cX/Q\nZ1UAx471hceGzMAsZAoLj82Ru25WrXIfN869Q4ewE9/4hvsTT9SdKUT9oNddRgUrjoLQHugG/AYo\nrzbtEWXFSUwqCAUkYoAZd1ic9friDG2TsHat+8SJ7nvvHXbk8MPdH33UfevW7ctkExbn+v5Ko0Qt\nCA2GylXMbE+grNqlpmZ/cqt6KhcYbzjAjDssLpQQfQcbN8I998CNN8LSpdCjRxhYb+jQ8BjQCMda\nCldsobKZ/aeZLSJcKvoHsJTwFDWRxvNoAWZWYXHtT/Q6PuGzDp9rV4+IX6CaTVV7ysrgf/4nPJPh\nvvvCvg8bFp7FMHUqXHppzfepR7PUIcpdRj8AjgDecvfuwCDg2URbJYWtqhjceiuMGhU+vEbVPax1\n5CGZBw6Evn23F4HKyjA/cGCNxbIa4jnl5zk3qK72XXUVLF4c7kB65BHo0AEuuigMidG/P6xbV++x\nbtJ2c+m4SKNFKQib3X0VUGJmJe7+d6B3wu2SQmYGu+8ePpgmTQrzkyaF+d13r3EpY9iw8AW3vDy8\nXF4e5ocNq7a+ykpYswYWLNheFPr2DfNr1tQ4U4i0PggfcqtX1/zgrCpiq1en/424ofaZwcknw4sv\nwtlnQ+fOMGtWeJBFx46hSNQ61rFsN+3jIk3TUMhAeJ7yrsAUQsB8K/BclIAi7kmhcoGJM8DcutW9\nd++awWnv3jXD1ca0L5d77mYbFj/7rPsJJ4Tl2rVzv/pq9w8+SHa7khOIsadyW+AzwtnEMMLdR9M8\nnDU0K4XKslOVlVBaun1+69aaQWpj5HoY25j2vfwy/PCH4RGfrVvDBReES03l5cluV1ITW6js7usJ\nPYoHuvuvgDuBz5veRJEYVVZCnz41X+vTp2m3DkUMvrNe587ms1muse3r1Qt++1t4883t19D23z8M\nwf3mmzt/b1O2K7mvoVMI4L+BOcDbmfkDgCejnH7EPemSkdRp61b3vfYKly569QrzvXqF+b32atxl\nozSHl27unsXLl4f37LJL6NV3+unu8+bVvax6NOclIl4yinI+PQL4N2BtpoAsAvZMojiJNIpZuNce\nYMCAMD9gQJhv0aJxlzKyCL4j8YhhbNTl4mxfly5wyy2h/8KYMTBzZgjljzsuBNFJHhfJLQ1VDOCF\nzM+XMj9bEMNYRo2ZdIYg9aqsdB85smbQOXJk07+xpjG8dNo9i1evdp8wwb1Tp7Dtr33NfcaMmutW\nj+a8Qoyh8k3AauAc4FLgEuANd/9ecmWqbgqVZac8D4LOqG3MhX3ZsAHuugtuuglWrIBDD4Vrr4VT\nT216WC/NKs7hr68BVhIGursI+DPw/aY1T5qkdhFvapgXdX1xbzeqCD2Qswo6k9iPKOuM2sZs9iVJ\nbdqEHs5vvx0Kw6efwhlnwCGHwK9+BZs3N297JHn1nToAXaOcYjTn1NRLRmk9AjJWcY9IGWfQmYQB\nA2r2J6jqbzBgwPZl0h7RM84QOJdD2y1b3KdP3x7Yd+3qPmWK+4YN6bVJIiGGUPn3Vb+Y2UNJF6ak\nTZsGw4fDsmXhi9ayZWF+2rS0W5YFj7mXaNT1xb3dqKL2QI4adCaxH1HXGbWNuRzalpaGwfJeegke\newz23Re+853Q+/nGG2Ht2vTaJvGor1KQCZFr/57m1JQzhEJ5dnDsvUSTCDrjlE0P5ChBZxL7kUQI\nnA+hbWWl+1NPuQ8eHPZ5993dr7vOfeXKtFsmtdDUUNnM5rt7n9q/p6kpoXJBDXvsMQeOUdcX93aj\nirsHchL7kdaxyRVz58KECWFAvTZtwlhJo0eHswhJXRyhci8zW2tm64Cemd/Xmtk6M8u7c8NUnx0c\np6pLEtU1JTyNur5sthunqstE1VUf1TRb2e5HnMcw23Y2tN1slktav37w8MPw+utw2mkweTLst18o\nDG+/nU6bJHtRTiNyZWrKJaPb6jMpAAANv0lEQVTUnh0cp7jD01wPOqtfLqq6TFR7PhvZ7kdazyLO\n9aA/infecb/4YvfWrd1LStz/67/cX3kl7VYVLWLsqVwQIg97nMviDk9zPegsKYH27aF3b5g3L8zP\nmxfm27fP/rJRNvsR9zGMKup2oy6Xlu7d4bbbYMkSuOIKePRR6NkTTjoJXngh3bZJ/aJUjVyZ1FM5\nI+7wNNeDztpnAk0Z0to9u/1No8dwrgf9jbFqlfu4ce4dOoR2Dhrk/uSTudnWAkTcz1TOBeqpnCUv\n8qAzDmkdw6jbzbf/xuvWhVPziRPhgw/g8MND7+cTTlDv5wTF2VNZ8lHVJYTqNERxUPsY1HdMsjmG\nUdcZtX25HPQ3Rbt24e6jJUvg9tvhww/DZaReveA3v4EtW9JuYXGLchqRK5MuGUWUy71d0xY1iE2r\n93OuB/1x27zZ/b773CsqQtu/9CX3qVPdN25Mu2UFBYXKRSyXe7umybMIYtPq/ZzrQX/cWrQIz3x+\n9dVw22qHDmEIgf32C/uzfn3aLSwuUapGrkw6Q8hSPvR2bW7ZBrFp9n5uaLvZLJcvKivdZ850Hzgw\nHMeOHd1vuMH944/TblleQ6GySD1cPZXzwnPPhWc/P/ZYyB4uuSScfe21V9otyzsKlSW62l8K6vuS\nEGUY6iS2G6eqSzrVxdGrON/C3Xxw1FHwxz+GwQyPPz48l6FbtzCg3rJlabeuIKkgFLtx42p+eFV9\nuI0bV3O5gQNrDhlRNaTEwIHJbjdO1a/vjxoV9mHUqJrX/3NhnVJTr17w29/Cm29u72G6//5w3nnh\nNYmNCkIxixqIRh2GOu7txi2JILZQwt18cOCBcOedYWykESNg+nSoqAgP7Zk/P+3WFYYoQUOuTAqV\nExA1EM1mGOo4t5uEJILYQgt388GHH7pfe637bruFfz9DhrjPmpV2q3ISCpUlsqiBaD4MQy3FZ82a\nMG7SpEmwciX07x96Px97rP49ZShUlmjc4bLLar522WU7XraprIQ+tR6J0adP/c833tl81WsKYiUO\n7dvDmDGwdGm47LhkCRx3XBiS+6GH8vCBJ+lJpSCY2Rlm9rqZVZpZg1VLEuIORx4Zxq4fOTL8jzNy\nZJg/8siaGcIXvwgvvxwCvq1bw8+XXw6vV/8fLkpYrCBWktCmTfj3+/bbcNddYdyk00+HQw6BX/0K\nNm9Ou4U5L60zhNeAU4GnU9q+ZMMs9CgFGDAgzA8YEOZbtEhu2G2RxmjVCs4/HxYuDHcntWoF554b\n7kz62c/gs8/SbmHuihI0JDUBTwH9oi6vUDkBlZXuI0fWDHdHjqw7JI26XBpDRovUp7LS/bHH3I88\nMvx73HNP9x/9yH3NmrRb1mzIh1DZzJ4CrnT3SEmxQuWERA13415OpDm5w9NPh2c/z5wZzkYvvTRc\nZvrCF9JuXaJSD5XN7Akze62O6aQs1zPczOaa2dyVK1cm1dzCFKVncdRwN5vlooTU2YgSUmeznBSn\nqkudjz8Oc+bA178ON9wQHp94xRXwz3+m3cL0RTmNSGpCl4ySM2BAzX4CVf0IBgzYvkzcQy1XVrof\nfnjNy0lVl5kOP7x4ny8suev1192/9S330lL3Vq3chw93X7w47VbFDg1/XcSi9izO9aGWPWJIHXU5\nkdoqKuDee2HRohBE33NP6BE9bBi89lrarWt+UapG3BNwCrAC2AR8CDwe5X06Q8hCNj2L4xxqOWr4\nHFUhPl9Yctd777lfeaV727bh39BJJ7k//3zarWoy8iFUzpZC5SzF3bM4Ko85VI66vri3K8Xr44/h\npz8NZ5mffAJnnhke8ZmnUg+VpZraRbc5inDVZaLqqo9WurP2NKV9HnOoXHX5p7pCeb6w5K499oCx\nY8Mw2xMnbu93U+iinEbkypSXl4zSCDurXy6qukxUez6J9sUdKhfb84VFEkLES0Yt0i5IBc2rhZ0Q\ngtjqQza4J3NJo6QkjO/SuzfMmxfm580LZwjt22+/rJJW+6KqL8yGaKF37eVEZOeiVI1cmfLyDCHN\nsLN2gFxfoBxn++IOlavWubP5bJcTKTIoVM4hnuNhZ9zty/X9FSkyCpVzhacYdtbeRl3bjLt9ae6v\niDSJCkKSqj4c0xjmOY1hqNPcXxFpMoXKSUor7IwaFsfdPoW7InlNGUJzqH23TnPcvVP923qV6h/U\nSbYvjf0VkXpFzRBUEAqZwl0RQaGyeALDUItIQVNBKETu0Z6VLCJSjQqCiIgAKgiFyQxmz95+VlBS\nsv1sYfZs5QgiUieFyoVMobKIoFBZsukxHKVHs4gUPBWEQpRNj+EoPZpFpCiop3IhitpjONeHvxaR\nZqUMoZBF6TGcTY9mEclL6qks0Sl8FiloCpWbophCVoXPIpKhglBbMYWsCp9FpBoVhOqqh6xVH35V\nH5irVxfeN+L6wudRo+oPn4vhuIgUKWUItRVjyKrwWaSgKVRuCoWsddNxEclLCpUbS88ErpuOi0jB\nU0GoTs8ErpuOi0hRUE/l6vRM4LrpuIgUBWUIddEzgeum4yKSl5QhNEXtDzl96AU6LiIFrbgKQtSe\ntuqRKyJFqHgKQtSetuqRKyJFKpWCYGY3m9mbZvaKmT1iZrsnusGoPW3VI1dEilgqobKZDQb+5u5b\nzOxGAHe/uqH3NSlUjtrTVj1yRaTA5E1PZTM7BTjd3Yc1tGyT7zKK2tNWPXJFpIDk011G5wMzEt9K\n1J626pErIkUqsYJgZk+Y2Wt1TCdVW+Z7wBZg2k7WM9zM5prZ3JUrVzauMVF72qpHrogUscR6Krv7\nMTv7u5l9GzgBGOQ7uW7l7lOBqRAuGTWqMVF72qpHrogUsbRC5SHAT4AB7h75a38sGUKUnrbqkSsi\nBSTXM4QpQDvgr2a2wMzuaJatRu1pqx65IlKEUhnczt33T2O7IiJSv1y4y0hERHKACoKIiAAqCCIi\nkqGCICIigApC02iYbBEpICoIjaVhskWkwKggNIaGyRaRApRKP4S8V31Ii1tv3T5UtobJFpE8lvrw\n19lo8tAVcdMw2SKSB3J96Ir8p2GyRaTAqCA0hobJFpECpAyhMTRMtogUIGUITaFhskUkDyhDaA4a\nJltECogKgoiIACoIIiKSoYIgIiKACoKIiGSoIIiICKCCICIiGXnVD8HMVgLLElj1F4CPElhvPtKx\nCHQcAh2HIN+PQ7m7d2poobwqCEkxs7lROm0UAx2LQMch0HEIiuU46JKRiIgAKggiIpKhghBMTbsB\nOUTHItBxCHQcgqI4DsoQREQE0BmCiIhkqCBkmNnNZvammb1iZo+Y2e5ptykNZnaGmb1uZpVmVvB3\nVdRmZkPM7P/MbLGZXZN2e9JiZneb2b/M7LW025ImM+tiZn83s4WZ/y9Gpd2mJKkgbPdXoIe79wTe\nAsak3J60vAacCjyddkOam5mVAj8DjgMqgLPMrCLdVqXmHmBI2o3IAVuA0e7+ZeAIYEQh/5tQQchw\n95nuviUz+zzQOc32pMXdF7r7/6XdjpR8FVjs7u+4++fAb4GTUm5TKtz9aeDjtNuRNnd/393nZ35f\nBywE9k23VclRQajb+cCMtBshzW5f4N1q8yso4P/5JTtm1g04FHgh3ZYkp6ieqWxmTwB71/Gn77n7\no5llvkc4TZzWnG1rTlGOQ5Gq65F3ug1PMLNdgYeAy9x9bdrtSUpRFQR3P2ZnfzezbwMnAIO8gO/H\nbeg4FLEVQJdq852B91Jqi+QIM2tJKAbT3P3htNuTJF0yyjCzIcDVwInuviHt9kgq5gAHmFl3M2sF\nnAn8IeU2SYrMzIC7gIXu/pO025M0FYTtpgDtgL+a2QIzuyPtBqXBzE4xsxXAkcCfzOzxtNvUXDI3\nFXwHeJwQHj7g7q+n26p0mNlvgNnAQWa2wswuSLtNKfk34FvANzKfCwvM7Pi0G5UU9VQWERFAZwgi\nIpKhgiAiIoAKgoiIZKggiIgIoIIgIiIZKghSNMxsa7VbBxdkhiLIdh27m9kl8bdu2/oPNrPZZrbJ\nzK5MajsiddFtp1I0zOxTd9+1ievoBjzm7j2yfF+pu2+NsNyeQDlwMvCJu09sTDtFGkNnCFLUzKw0\n8yyMOZlnYVyUeX1XM3vSzOab2atmVjXq6Y+AL2XOMG42s4Fm9li19U0xs3Mzvy81s+vN7BngDDP7\nkpn9xczmmdksMzu4dnvc/V/uPgfYnPjOi9RSVGMZSdHbxcwWZH5f4u6nABcAa9z9MDNrDTxrZjMJ\no56e4u5rzewLwPNm9gfgGsJzM3oDmNnABra50d2/lln2SeB/3H2RmR0O3AZ8I+6dFGksFQQpJp9V\nfZBXMxjoaWanZ+bbAwcQBrqbYGZHA5WEYbD3asQ2p8O20TKPAn4XhscBoHUj1ieSGBUEKXYGXOru\nNcZsylz26QT0dffNZrYUKKvj/Vuoeem19jLrMz9LgNV1FCSRnKEMQYrd48DFmSGOMbMDzawt4Uzh\nX5li8HVC0AuwjjAIYpVlQIWZtTaz9sCgujaSGUN/iZmdkdmOmVmvZHZJpHF0hiDF7k6gGzA/M9Tx\nSsIdPtOAP5rZXGAB8CaAu68ys2czD5+f4e5XmdkDwCvAIuClnWxrGHC7mX0faEl4ROfL1Rcws72B\nucBuQKWZXQZUFPJDWSR36LZTEREBdMlIREQyVBBERARQQRARkQwVBBERAVQQREQkQwVBREQAFQQR\nEclQQRAREQD+HwwJo8DixpVTAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa27d358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 计算分类正确率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.82\n"
     ]
    }
   ],
   "source": [
    "s = np.dot(X, w)\n",
    "y_pred = np.ones_like(y)\n",
    "loc_n = np.where(s < 0)[0]\n",
    "y_pred[loc_n] = -1\n",
    "accuracy = len(np.where(y == y_pred)[0]) / len(y)\n",
    "print('accuracy: %.2f' % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 注意："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过实例验证，优化后的PCA算法不一定每次都能找到最优分类线。在足够的迭代次数下，使用一般的PCA算法，竟一般都能得到最佳的分类线，即使开始的时候，分类线会有所振荡。但最终都基本能寻找到最佳位置附近。这也是PCA算法的特点之一。下面是使用一般PCA算法处理非线性可分样本实例的效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "直线初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XmYFNW9//H3l01Wd+NPRQZ3RQRk\nEZcoJBrEJJdIokaCa1SMomDifk0C11x4orhhiAtxuwbi1SgaNaJIjIqKCYtoUIwYVORqFFAWQRCY\n7++P0w09wyzV09VdvXxez1PPUD3VVacrpr9T9alzjrk7IiIizZJugIiIFAcVBBERAVQQREQkRQVB\nREQAFQQREUlRQRAREUAFQUREUlQQREQEUEEQEZGUFkk3IBs777yzd+7cOelmiIiUlDlz5ixz910a\n266kCkLnzp2ZPXt20s0QESkpZvZBlO10y0hERAAVBBERSVFBEBERoMQyhLps2LCBJUuWsG7duqSb\nUpJat25Nx44dadmyZdJNEZGElXxBWLJkCR06dKBz586YWdLNKSnuzvLly1myZAl77bVX0s0RkYSV\n/C2jdevWsdNOO6kYNIGZsdNOO+nqSkSAMigIgIpBDnTuROIzeTJ07gzNmoWfkycn3aLslPwtIxGR\nYjB5MgwbBmvXhvUPPgjrAEOHJteubJTFFULSmjdvTo8ePejatSsnn3wya9P/RWTh3HPP5a233gJg\n7NixNX535JFHxtJOEcmfa67ZUgzS1q4Nr5cKFYQYtGnThnnz5jF//nxatWrFHXfckfU+7rrrLrp0\n6QJsXRBeeeWVWNopIvmzeHF2rxejiisI+b7Hd/TRR/Puu+8CcNNNN9G1a1e6du3KLbfcAsCaNWv4\nzne+Q/fu3enatSsPPvggAP3792f27NlcddVVfPnll/To0YOhqevM9u3bA/DDH/6Qp556avOxzjrr\nLB555BE2bdrE5ZdfTp8+fejWrRt33nlnvB9KRBrVqVN2rxcldy+ZpVevXl7bW2+9tdVr9Zk0yb1t\nW3fYsrRtG17PRbt27dzdfcOGDT5o0CC/7bbbfPbs2d61a1f/4osvfPXq1d6lSxefO3euP/zww37u\nuedufu+KFSvc3b1fv34+a9asGvurvf8pU6b4GWec4e7u69ev944dO/ratWv9zjvv9F/96lfu7r5u\n3Trv1auXL1q0KHL7szmHIlK3fH2/xAGY7RG+YyvqCiFf9/jSf9H37t2bTp06cc455/DSSy8xePBg\n2rVrR/v27fn+97/PjBkzOOSQQ5g+fTpXXnklM2bMYLvttot8nBNOOIHnnnuO9evXM3XqVI455hja\ntGnDtGnTuP/+++nRowd9+/Zl+fLlLFy4MLcPJSJZGToUJk6EqiowCz8nTiydQBkq7CmjfN3jS2cI\nmUJR3tr+++/PnDlzeOqpp7j66qsZMGAAv/zlLyMdp3Xr1vTv359nnnmGBx98kCFDhmw+1m9+8xuO\nP/743D6IiORk6NDSKgC1VdQVQiHv8R1zzDE89thjrF27ljVr1vDoo49y9NFH89FHH9G2bVtOO+00\nLrvsMubOnbvVe1u2bMmGDRvq3O+pp57Kvffey4wZMzYXgOOPP57bb79983veeecd1qxZE/+HEpGy\nVlFXCGPG1HxOGKBt2/B63Hr27MlZZ53FYYcdBoTHSg899FCeeeYZLr/8cpo1a0bLli25/fbbt3rv\nsGHD6NatGz179mRyrdR7wIABnHHGGQwaNIhWrVpt3vf7779Pz549cXd22WUXHnvssfg/lIiUNavv\n1kYx6t27t9eeIGfBggUcdNBBkfcxeXLIDBYvDlcGY8aU9iVeHLI9hyJStwsvDLnBpk3QvHn4A/S2\n25JuFZjZHHfv3dh2FXWFAKV/j09EitOFF0LmBf+mTVvWi6EoRFFRGYKISL5MnJjd68VIBUFEJAab\nNmX3ejFSQRARiUHz5tm9XoxUEEREGhFlyJv0yKZRXy9GFRcqi4hkI+qw1unguBifMooqsSsEM9vT\nzP5qZgvM7E0zG5lUW3JlZlx66aWb12+44QZGjx4d+3E0LLZI4WUz5M1tt8HGjWEko40bS6sYQLK3\njDYCl7r7QcDhwHAz65L3o9budxFDP4xtttmGKVOmsGzZspz31RANiy1SeOUwrHVUiRUEd//Y3eem\n/r0aWADskdeDjh4NP/3pliLgHtZz/Gu+RYsWDBs2jJtvvnmr3y1dupQf/OAH9OnThz59+vDyyy9v\nfv1b3/oWPXv25Pzzz6eqqmpzQTnxxBPp1asXBx98MBNTz6xpWGyRZJTFsNZRRRkSNd8L0BlYDGxb\nx++GAbOB2Z06ddpqWNfIQzdXV7uPHBnGpB05su71JmrXrp2vXLnSq6qqfMWKFT5u3DgfNWqUu7sP\nGTLEZ8yY4e7uH3zwgR944IHu7j58+HAfO3asu7tPnTrVAV+6dKm7uy9fvtzd3deuXesHH3ywL1u2\nbPNxah/XPfdhsTX8tUj9inlY66iIOPx14qGymbUHHgEucfdVtX/v7hOBiRCGrsjhQJD+C378+LAA\njBwZXs9xsvltt92WM844g1tvvZU2bdpsfn369Ombp8YEWLVqFatXr+all17i0UcfBWDgwIHssMMO\nm7e59dZbN//uww8/ZOHChey00071HvuEE05gxIgRrF+/nqeffrrGsNhvvPEGDz/8MAArV65k4cKF\n7LXXXjl9VpFKkg6OK2HIm0QLgpm1JBSDye4+pQAHDF/+6WIAsRSDtEsuuYSePXty9tlnb36turqa\nmTNn1igSUP/w2M8//zzTp09n5syZtG3blv79+7Nu3boGj6thsUXyq1KGvEnyKSMD7gYWuPtNBTlo\nOjPIlJkp5GjHHXfklFNO4e6779782oABA5gwYcLm9fS8CV//+td56KGHAJg2bRqff/45EP6K32GH\nHWjbti1vv/02r7766ub3alhsEcmnJJ8yOgo4Hfimmc1LLd/O29HSxWD8+HCbqLo6/Bw/PtaicOml\nl9Z42ujWW29l9uzZdOvWjS5dunDHHXcAMGrUKKZNm0bPnj2ZOnUqu+22Gx06dGDgwIFs3LiRbt26\n8Ytf/ILDDz98877Sw2IPreNPlQEDBvDiiy9y3HHH1RgWu0uXLvTs2ZOuXbty/vnns3Hjxlg+p4iU\nn8oa/nr0aFixYsttonSR2H77nJ80ytb69etp3rw5LVq0YObMmVxwwQVbzbpWKBr+WqQ4xTVcv4a/\nrsvo0aEIpDODdKYQU4aQjcWLF3PKKadQXV1Nq1at+N3vflfwNohI8YraQzpOlVUQYOsv/wSKAcB+\n++3Ha6+9lsixRaT4NdRDOl8FoSwGtyul217FRudOpDgl0UO65AtC69atWb58ub7YmsDdWb58Oa1b\nt066KSJSSxI9pEv+llHHjh1ZsmQJS5cuTbopJal169Z07Ngx6WaIRFJJc6KPGVMzQwBo2za8ni8l\nXxBatmypnrciFSCJkDVJSfSQLvnHTkWkMnTuHIpAbVVV8P77hW5NaYn62GnJZwgiUhkqaRjqpKgg\niEhJqKhhqBOigiAiJWHMmBCqZsp3yFppVBBEpCQMHRrmK66qCv1Jq6rCejkGykkp+aeMRKRyVMow\n1EnRFYKIiAAqCCIikqKCICKJmzw59DNo1iz8nDy5uPZX7MeNizIEEUlU3D2Qk+rRXA49qdVTWUQS\nFXcP5KR6NBdzT2r1VBaRkhB3D+SkejSXQ09qFQQRSVTcPZCT6tFcDj2pVRBEGlDqIWEpGDMGWras\n+VrLlk3vgZxUj+Zy6EmtgiBSj3RI+MEHYSrudEioohC/OGe2TapHczn0pFaoLFKPYg4Jy4nOc/4p\nVBbJUTmEhKVA57l4qCCI1KMcQsJSoPNcPFQQROpRDiFhWjH33C2n81zy3L1kll69erlIIU2a5F5V\n5W4Wfk6alHSLsjdpknvbtu4hGg9L27b5/yzZHLccznMxA2Z7hO9YhcoiZU49d0WhsogA6rkr0akg\niJQ59dzN0VdfwSefJN2KglBBECmgJIZ5LoWeu0XZI3z1arjpJthnH/jJT5JuTWFECRqKZVGoLKUs\n7nC3FELbKMdNKvSu17//7f6f/+m+/fahMf37u0+dmlBj4oFCZZHiUi7DPMetaD7Hv/4FN9wA994b\nbhMNHgxXXgmHHVbARuRH1FA50QlyzOwe4LvAp+7eNcm2iORbuQzzHLfEP8ecOXDddfDII9CiBZx5\nJlx6KRxwQIEaUDySzhDuAwYm3AaRgiiXYZ7jlsjncIdnn4XjjoPeveGZZ+Dyy8MlycSJFVkMIOGC\n4O4vAp8l2QaRQok73B0zJoSwmZo1yy0sTiLcLWjovXEj/O//Qq9eMGAAvPUWXH89fPgh/PrXsNtu\neTho6Uj6CkGkYsQ9PPLLL0N1dc3XqqvD602R1HDfBRk2eu1a+O1vYf/9YciQsH7XXfDee+HKYNtt\nYzxY6Uo8VDazzsCT9WUIZjYMGAbQqVOnXh/UlT6JVKAWLWDTpq1fb948/CGcraIJd+P02WehENx6\nKyxbBocfHoLiQYO2vrwqYyURKkfh7hOBiRCeMkq4OSJFo65i0NDrjUk83I3T4sWhD8Fdd8GaNfCd\n78AVV8DRR+c2+06ZK/qCICJ1a968/iuEpujUqe4rhJIKqf/xDxg3Dh54IKwPGRJuCR1ySLLtKhGJ\nXjOZ2QPATOAAM1tiZuck2R4pbfkIRC+8MNyaMQs/L7wwt+3iNGxYdq83pmSHoXaHF18MVwHdusGU\nKXDRRaFfwf33qxhkI0rvtWJZ1FNZ6pOP3q4XXFBzf+nlgguatl3cJk1yb9685jGbN8/tM5fUMNSb\nNrlPmeJ++OHhw++8s/u117ovX550y4oO6qkslSQfgWjU0DbucDeqsgyBo1i/HiZNCreG/vlP2Gsv\nuOwyOOusrS9xBCijUFkkinwEolFD27jD3ajKKgSOYuVKuPNOuOUW+PhjOPTQkBWcdFKoypKzynnu\nSspaPnq71hfO1n496nZxK5eeyo36+GO46qrwwa68Erp0gWnTwpATp56qYhAjFQQpC/kIRKOGttmE\nu3EG3yU/vHRj3nkHzjsvNHjcOBg4EGbPhunT4Vvf0uOj+RAlaCiWRaGyNCQfgegFF2wJbps3rzso\nnjTJvUWLmuFuixZbHz8fwXdJDi/dmL/9zf373w8faptt3H/yE/eFC5NuVUlDobJIYUQNdzW3cQPc\n4emnw7hCzz8P228Pw4fDxRfDrrsm3bqSp1BZpECihrua27gOGzbAQw+FQvDGG9CxY+hhfO650KFD\n0q2rOMoQRHIUNdzV3MYZ1qwJ4wvttx+cdlp4Pve++0Jnsp/+VMUgISoIUnHiDlijhrua25gwwNzo\n0eF+1ciRsOee8MQTYciJM8+EVq1iPqBkJUrQUCyLQmXJVb4C1qiBdsXObfzee+4XXeTepk3Y8aBB\n7i+/HMOOJQoUKotsrSQC1oTk5dy8/nrIBx58MFx2nHZaGGzuoINyaKlkS6GySB2KOmBNWGznxj08\nKXTddWFqyvbtQy4wcmQIjaVo1ZshmNkhZvaqmX1oZhPNbIeM3/29MM0TiVdRBqxFIudzs2kTPPww\n9O0L3/wmzJsHY8eG6SnHjVMxKAENhcq3A6OBQ4B3gJfMbJ/U71rmuV0ieZGvYDeJ4a/j1uRzs25d\nmPPyoIPg5JPh88/DmEPvvw9XXx36FEhpqC9cAObVWv8GsBA4HJgbJaCIe1GoLHGIO9hNavjrfMjq\n3Hz+ufvYse677ho+cO/e7n/8o/vGjQVqrURFrqGymb0OHOPuKzNe6wY8Auzo7jvlt1RtTaGyFKOk\nhr9OzP/9H9x8c7gK+OILOP74MOhc//4aX6hIRQ2VG7pldB1Q41EAd38DOBaYklvzRMpHUsNfF9yC\nBfDjH4f5B265JUxU/9prYciJb3xDxaAM1PuUkbv/oZ7XFwPn5a1FIiUm7rmNi84rr4Qnhh5/HNq0\ngfPPh5/9LBQGKSvqqSySo6SGv87H/jarroYnn4Sjj4ajjoKXXoJRo0JHhd/8RsWgXEUJGoplUags\nxSrqMNlx9gTOS8/i9evd77vP/eCDww47dXIfP979iy9y2KkkDfVUFikucfcEjnV/q1fD734XwuIl\nS+CQQ0JQfMop0FJPmZe6OELl9I72N7O/mNn81Ho3M/t5HI0UqSRx95KOZX+ffgo//3nofXbppbDv\nvvDUU2HIiaFDVQwqTJQM4XfA1cAG2Pyk0an5bJRIDbWvYkvoqjZT3L2kc9rfv/4FF1wQLifGjg09\ni199Ff76VzjhBD0xVKGiFIS27l57qIpyfLpaYnLcceH7JL0cd1zd20UKREePDuPgpIuAe1gfPTov\nbW9SGyMaM2br0Z1btWp6L+km9SyeMwd++EPYf3+45x44/fTwOOkjj0DfvqU597LEp7GQAZgK7EOq\ndzJwEjA1SkAR96JQufgde2zdvXaPPbbmdpEC0epq95Ejwy9Hjqx7PY/yEQK3bFlzfy1b5n9OZa+u\ndp82bcv/ONtu637lle4ffbTVvkpq7mWJjIihcpSCsDcwHVgL/B/wElAVZedxLyoIxa+uYpBeMlVV\n1b1NVVWtHWYWgfRSgGKQVRsT2l+jNmxwf+AB90MPDQfabTf36693X7myONonBRO1IDT4lJGZNQNO\ncveHzKwd0MzdV+ftcqUResqo+DV06znzP7VmzeqOAszCI/BbvbFZxt3N6uqC3OPOqo0J7K9eX34J\n994LN94IixbBAQeEOQhOOw222Sb59knBxfKUkbtXAxel/r0myWIg5SVyIOqpzCBTZqaQR0UVAtcn\n8zx89hlce20IiocPh699DR59FN56C845p8FikLf2SUmJEio/a2aXmdmeZrZjesl7y6ToRAkcjz22\n7vfWfj1SIJouBuPHc0+HkTSjmns6jITx4wtSFMaM2fqpy5YtCxwCNyQduH/wQfjZqVPoTbz99vDC\nC2HIiRNPrHl1Vcj2Selp7J4S8F4dy6Io96PiXpQhJCebwLF2sFw7UM7cZ2OB6OuDR/mEFiMdqlP7\nq/YJLUb664NHxfjp6m9fq1Y1P0urVgUIgaOornYfOjQ0qlkz9xYt3A88MOeMJak5nyW/UE9liVNS\ncxGH4zqQmRk4VVWW9zmQi3L+ZXeYMSPMU/znP4extzPH2B45MvQ2Vj8CyRA1Q2i0IJjZGXW97u73\nN7FtTaaCkJykAsckg86iClmrq8Noo9ddFzqQ7bwzjBgROpftskvN7VQMpJbYhq4A+mQsRxOm1RyU\nU+tSzGygmf3TzN41s6vi2KfkR1KBY5JBZzhG7YrghQ1Z16+Hu++GLl1g8GD45BOYMCFcuvz85/Df\n/11z+1yzldrvLaE7CBKDKPeVMhdgO+DxbN9Xx36aA/8i9HNoBbwOdGnoPcoQ8iPKfeOkOi1lc9y4\n73+/PniU39qsZn5xa7PC5Be+cmXoM7D77uFD9+gR+hRs2BB+n49Oe6NG1Xxvep+jRsX1qSQhxNUx\nbas3QEtgQbbvq2M/RwDPZKxfDVzd0HtUEOKX5BduNm0seMGqrvYFx4cv2JsJReFmwvqC4/PYMe6j\nj0Iv4m233ZLIT5tW9/Hi/AJPuFe45FdsBQF4Ang8tTwJLAKui7LzRvZ7EnBXxvrpwISG3qOCEL9y\n6Z2aj89R1WlLEUgvNzPSqzrl4cvxn/90P++88BhTs2buJ5/sPmtW4++r/UWdyxd3gr3CJb+iFoQo\noXK/jNWNwAfuvqTpN6k27/dk4Hh3Pze1fjpwmLtfXGu7YcAwgE6dOvX6oK7HPqTJiio4zUE+PkfY\np+MZUZtRjZnFd27+/vcQFD/6aBjp7uyztwxDnQRPple45FecofK33f2F1PKyuy8xs+tiaOMSYM+M\n9Y7AR7U3cveJ7t7b3Xvvkvk0hcQi0d6ptb/BG/njpCH5+Byd9nRupmYv6Zv5KZ32zDFodYepU8PE\n9H37wnPPwdVXh6D49tvzUwyinGtPrle4FInGLiFIjXJa67U3olx+NLLfFoTbT3uxJVQ+uKH36JZR\n/BIb4TKL+99xZwhRRwiNPUP46qtwsG7dQgP32MP9xhvdV63Kfl/ZiHKulSGUNXLNEIALgH8Aa4A3\nMpb3gElRdt7oweHbwDuEp42uaWx7FYT8KHhYnMWXT9xf9FkVwFGjfMHxITMwC5nCguObENp+8UWY\nlzgddHTpEuYtXr8+u/00RTZf9HrKqGzFURC2AzoDDwBVGcuOUXacj0UFoYxEDDATH4I6l9B26dLw\nZbrTTuEgRx3l/vjj7ps2Na3xTZVNWBxnSC1FI2pBiDx0hZl9DWidcaupiTPBNp16KpcZbzzALMkh\nqN9/Pww9fffdYSjqQYPgiivgqKNiOkATRDjXUr5iC5XN7D/MbCHhVtELwPuEWdREms6jBZhZhcW1\nv9Hr+IbPOnyuXT0a+gMqPTH9vvvCnXfCqafCm2/Cn/6Uv2IQpX0Rz3Xsx5XS09glBCHs3Ql4LbX+\nDWBilMuPuBfdMioT+cgQ+vULvXnTt2M2bQrr/frV2CzbDCFSGPvcc+7HHx921r69+6WXun/4YW7n\nKIqkwmJlDSWHGDumzfYthaFZ6t9/j7LzuBcVhDIS51NG6S//9BAPda1ns7/M9tT3Rbphg/vDD7v3\n6RNe23VX97Fj3T/7LNczE01SYbGeRipJcRaE6UB7YAIhYB4PvBJl53EvKghlJs4AM7MIpJc6ikHW\n7asdxg4f7n7HHe777RfW9903rH/5ZdOPE2f7ChEWq0dzyYlaEKL0VG4HfEnIG4YSnj6a7O7L47tx\nFY1CZWlQdTU0b75lfdOmyLOF1ctrhbG77hpGHO3dG668MoxAmnnMQqvdvkKFxUkdV5oktlDZ3dcQ\nehT3d/f/Ae4Cvsq9iSIxqq6Gnj1rvtazZ26PDrnDeefVfK11a5g+PQw5cdJJ2ReD2n+A1fcHWZTt\nPKGexUkdV/IuylNG5wEPA3emXtoDeCyfjRLJSnU17L57eMqne/dwZdC9e1jfffemFYW33oKuXcOj\no2YwZAj86EdheIknnmhaO9NzIKe/ONNfrKNHZ79d+rXx48MsadXV4We+55tO6rhSEC0ibDMcOAz4\nG4C7L0z1SRApDmZhKkmAfv3Cer9+oSC0aJHdrYyZM8Ngc3/6U3hv9+4wZQrsvXf4sttllzCJfba3\nR9xhxYrwxQlhmsvML1b3sM+o25mFdmROmXnzzeE9TWlfVEkdVwqjsZAB+FvqZ/qx0xbEMJZRUxaF\nylKv6mr3ESNqBp0jRkQLOjdtcn/iCfevfz28b8cd3X/5S/dPP00mjC2FnsXq0VxSiDFUvh5YAZwB\nXAxcCLzl7tfkr0zVTaGyNCjboPOrr+CBB2DcuNCBrFOnMPT0OedAu3bJtlGhrcQozuGvrwKWEga6\nOx94Cvh5bs2TnEQNJuPeX9zHjSpCD+Ssgs7Vq8Ntjn32gbPOCl+8v/89vPtumLi+KcUgzhBYoa0k\npb5LB6BTlEuMQi653jJKagrIWMXdSzTq/pLqnRqlB3LUzlKffOJ+9NHu22wTftevn/uf/xxuLeXy\nOeLsMayOX5IHRLxl1NAVwuYniczskXwXpnybPBmGDQsPibiHn8OGhddLhmcEjum/GNOB44oV2f8F\nGXV/cR83qupqWLkS5s2DXr3Ceq9eYX3lyi1XCvUFnSNHhtcXLYILL4SqKpgxA9avhx/+EP76V5g2\nDW69temfI+q5aayN6dtBUbcTyYf6KgWpELn2v5NccrlCKJe5g2PvJZqPoDNO2fRArt2W2bPdTzkl\nzFHcqpX7uee6L1gQ/+fIRwis0FZiRK6hspnNdfeetf+dpFxC5XKZOxiIP3CMur+4jxtVNj2Q3UPH\nseuvDz+33RYuuCD8hb3bblu2iftzJHVuRCKII1TubmarzGw10C3171VmttrMVsXX1MJIdO7gOKVv\nSWSqL3Cs/Vp92xRz0Jm+TZQpffso08aN8OCDYUiJAQPCU0PXXQeLF8Ovf12zGGTzOeI8h9mIctxs\nthOJIsplRLEsudwySmzu4DjFPcJlsQedUUYxXbvW/be/dd977/D6AQe433WX+7p1uZ0/9+IfXlrD\nUEtERLxlFKWnclkYOjT8vOaa8Edjp04wZsyW10tC1F6iHnNv16R6pzZrBtttBz16wJw5YX3OnHCF\n0LYtjB0bAuGlS6FvX7jhBvje9+q/nZTN54j7HEYV9bhRtxPJRpSqUSyLeiqnRAkcyynozAyQFy8O\nn6Ndu/CZvv1t9xdeyK4t2XzeYh5eWsNQS0TEPadyMVBP5Sx5GQWd8+eHHsV/+ENYHzIELr8cDjkk\nv8dN6hxGPW45/W8seRNnT2UpRV4GvV3dQ7+B7343fPE//DAMHw7/+hfcf3/Ti0Htc1DfOcnmHEbd\nZ9T2FXPQL+UrymVEsSy6ZRRRqfd23bTJ/dFH3Y84IrR5553dr73Wfdmy3PcdNYgt9ikqS/1/Yyko\nFCpXsFIdonj9+tB1fNw4ePtt2GsvmDABzj47hMi58iyC2LgD/KiKPeiX8halahTLoiuELJVKb9eV\nK92vv95999198yOlDzwQJrKPW7ZBbNwBfjbtbOy42WwnFQ2FylL0Pv44PDZ6++1hbKJjjw3zFB93\nXH7/wnX1VJbKolBZoqv9R0F9fyREGYY6infeCSMLdu4chpgYMABmzYJnn4VvfSv/xSAfvYoV7koZ\nUEGodFHn+e3fv+aQEekhJfr3j36s9MT0Bx4I99wD++8fsoKHHgr7quu4cUp/tjjnA87HPkUSooJQ\nyTID0YaGbo46DHV9x3j6afjGN0Jv4r/8Ba66KgTF8+fDb39b/3Hjlo+hpTVctZSTKEFDsSwKlfMg\naiCazTDU7iEQnjTJvVu3sO0ee7jfeKP7qlXZHTcf8hHEKtyVIoZCZYksaiAaZRjqNWvC7aAbbwyz\nEB10EFxxBfzoR9CqVdOOKyI5Uags0bjDJZfUfO2SS7a+bVNdDT1rTYnRs+eW20XLloX7/1VVYV7i\njh3h8cfDbaEzz6y7GCiIFSkuUS4j4l6Ak4E3gWqgd9T36ZZRzKqr3fv2DbdrRowI6yNGhPW+fbfc\n9ti0yX3XXcPr3buH9e7dfXMgU33yAAAMoUlEQVQv4osucm/TxjcPP33yyYUfMlpE6kUMcyrn03zg\n+8CLCR1fsmEGLVKd2vv1C+tduoT1ZcvgzjvDHMXz58PAgfDHP8Yzv7CIFFaUqpGvBXgeXSEkK/Oq\nIL2krxZqb3fxxTW3a9nS/Wc/c//ww5rbJTFktIjUi1IIlc3seeAyd4+UFCtUzpPGwt1Nm+Cxx8KU\nlLNmbXl9+XLYccfs9yciBZV4qGxm081sfh3L97LczzAzm21ms5cuXZqv5panKD2LGwp3162DiRPD\nk0InnRR6GGe69tq6ezlHCamzUdcxctlOROoW5TIiXwu6ZZQ//frV7CeQ7kfQr9+WbRoKd488ckuQ\n3KtXmJksypDMUULqbGh+YZGcUeShsuRT1J7FtcPdjz4K4XGrVvDKK9C9e+hZPGsW9OlT+BDYI/ak\njrqdiDQsStWIewEGA0uA9cAnwDNR3qcrhCxk07P4zTfdzz47hMTNmrkPGeI+d+7W20UdCjpKSB2V\n5hcWyRmlECpnS6FylhrrWTxzZgiK//QnaNMGfvxjuPTSMDFNLjzmUDnq/uI+rkiZSDxUlgy1i24h\ninD6NlGmXr1g40Z48kk45hg48sgwZ/EvfhGGmZgwIRSDXNrnMYfK6ds/mTS/sEheqCDkW9ThpeOU\nmRn06BGuDLp3D+sdOsB//EcoALfcAuefD6tWwc47594+dzjiiDDpzYgRoR0jRoT1I47I/ss5Mwto\naGjpqNuJSINUEPIpqbCzWTPYbrtQDF54IRzvs8/C75o3h9//Ht59N3xZr11bvGFs1B7N6vksEo8o\nQUOxLCUZKicVdn7yifs117jvsEM4Zr9+7k88kf8wNu5QOb3Phtaz3U6kwqBQuYgUMuxctAhuuAHu\nvRfWr4cTTwzzFPftW7j2KdwVKSoKlYtFocLOuXPh1FNhv/3g7rvhtNPgrbdgypQtxaCuY8bdPoW7\nIiVLBSGf8h12usP06WFi+l69YOpUuOwyeO892GMPuOOOhsPsuNuncFekpLVIugFlrb6wE3ILOzdu\nhEcegeuvD1cGu+0W+hOcf34IkzPDbAjHzPyidg/Hjrt9+fq8IlIQyhAKIf0FXN96VF9+CffdFzKC\nRYtg//3h8svh9NNhm222Pma6CKRlflHno3352p+I5CRqhqCCUAo++wxuuy08z790acgErrwSvve9\nrec0zqRwV0RQqFwePvwQfvYz6NQp9Cbu0yf0K5g5EwYPbrwYxD0MtYiUNRWEYvTmm2Fi+r33DlcF\ngwfDG2/An/8chpxo7K/8uHsMi0hFUKhcLNzhpZdCUPzkk9C2LQwfHnKAqqqkWyciFUBXCEmrrg6j\njR51VPjr/9VX4b/+CxYvDmMNNaUYmIXbSumrgmbNtlwtzJypHEFE6qRQOSnr18PkyTBuHLz9NnTu\nHPoQnH12uDqIg0JlEUGhcvFatSo8Nrr33nDOOdC6NTzwACxcGG4RxVkMovYYTmJ4bhEpOioIhfLv\nf8PVV4cnhi6/HA48EJ55ZsuQEy1ijHOy6TGcxPDcIlKUFCrn28KF4Yrgf/4HNmyAH/wArrgCejd6\n9dZ0UXsMR+3RLCIVQRlCvsyaFYaTmDIlTFp/1lkhI9h338K1IUqP4Wx6NItISVJP5SS4h9tA110H\nzz8f/hq/8MLwdM+uuybduvopfBYpawqVc5FtyLpxI/zhD3DooXDCCeE20Y03hkdHx4wp/mKg8FlE\nUEHYWjYh65o18JvfhNtAQ4fCV1+FiWkWLQpDTnToUMiWZ0/hs4hkUEHIlBmyNjTH8LJlofNYVVW4\nHdSxIzz+OMyfH7KCVq0S/RiRRZ2LOOp5EZHSFmWezWJZCjKnckNzDL/3nvvFF7u3bRteHzTI/aWX\n8t+mfIsyF3FSc0OLSM7QnMo5qB2yvvZa6FH84IPhr+bTTgt9Cbp0yX9bionCZ5GSpFC5qeoKWQ89\nNNwSGjkyTE95772VWQw0V7JIWVNByOS+JVRNPxnUpk34OXRo6GDWsWNy7UuK5koWqQjqqZy2bh38\n/vdhwDmA9u3DEzRnnAH/+Z+VPSew5koWqQjKEFasgDvuCH/t/vvf0KtXGFriBz+A5s3DNhrCIdBc\nySIlKWqGoCuEv/wlDDo3YABMmgTf/ObWX3L60gt0XkTKWmUVhLr+wj3xRJg3D7p3b3g7ffmJSJmr\nnFC5vp62v/pVzWKgHrkiUqESKQhmNs7M3jazN8zsUTPbPq8HjNrTVj1yRaSCJRIqm9kA4Dl332hm\n1wG4+5WNvS+nUDnqMM8aDlpEykzJDH9tZoOBk9x9aGPb5vyUUdSetuqRKyJlpJR6Kv8YmJr3o0Tt\naaseuSJSofJWEMxsupnNr2P5XsY21wAbgckN7GeYmc02s9lLly5tWmOi9rRVj1wRqWB5e+zU3Y9r\n6PdmdibwXeBYb+C+lbtPBCZCuGXUpMZE7WmrHrkiUsGSCpUHAjcB/dw98p/9sWQIUfoXqB+CiJSR\nYs8QJgAdgGfNbJ6Z3VGQo0btaaseuSJSgRLpqezu+yZxXBERqV8xPGUkIiJFQAVBREQAFQQREUlR\nQRAREUAFITd19XIWESlRKghNpWGyRaTMqCA0hYbJFpEyVFkzpsUlc0iL8eO3DJWtYbJFpIQlPvx1\nNnIeuiJuGiZbREpAsQ9dUfo0TLaIlBkVhKbQMNkiUoaUITSFhskWkTKkDCEXGiZbREqAMoRC0DDZ\nIlJGVBBERARQQRARkRQVBBERAVQQREQkRQVBREQAFQQREUkpqX4IZrYU+CAPu94ZWJaH/ZYinYtA\n5yHQeQhK/TxUufsujW1UUgUhX8xsdpROG5VA5yLQeQh0HoJKOQ+6ZSQiIoAKgoiIpKggBBOTbkAR\n0bkIdB4CnYegIs6DMgQREQF0hSAiIikqCClmNs7M3jazN8zsUTPbPuk2JcHMTjazN82s2szK/qmK\n2sxsoJn908zeNbOrkm5PUszsHjP71MzmJ92WJJnZnmb2VzNbkPr/xcik25RPKghbPAt0dfduwDvA\n1Qm3Jynzge8DLybdkEIzs+bAb4ETgC7AEDPrkmyrEnMfMDDpRhSBjcCl7n4QcDgwvJz/m1BBSHH3\nae6+MbX6KtAxyfYkxd0XuPs/k25HQg4D3nX3Re7+FfC/wPcSblMi3P1F4LOk25E0d//Y3eem/r0a\nWADskWyr8kcFoW4/BqYm3QgpuD2ADzPWl1DG/+eX7JhZZ+BQ4G/JtiR/KmpOZTObDvy/On51jbv/\nKbXNNYTLxMmFbFshRTkPFaquKe/0GJ5gZu2BR4BL3H1V0u3Jl4oqCO5+XEO/N7Mzge8Cx3oZP4/b\n2HmoYEuAPTPWOwIfJdQWKRJm1pJQDCa7+5Sk25NPumWUYmYDgSuBQe6+Nun2SCJmAfuZ2V5m1go4\nFXg84TZJgszMgLuBBe5+U9LtyTcVhC0mAB2AZ81snpndkXSDkmBmg81sCXAE8GczeybpNhVK6qGC\ni4BnCOHhQ+7+ZrKtSoaZPQDMBA4wsyVmdk7SbUrIUcDpwDdT3wvzzOzbSTcqX9RTWUREAF0hiIhI\nigqCiIgAKggiIpKigiAiIoAKgoiIpKggSMUws00Zjw7OSw1FkO0+tjezC+Nv3eb9H2hmM81svZld\nlq/jiNRFj51KxTCzL9y9fY776Aw86e5ds3xfc3ffFGG7rwFVwInA5+5+Q1PaKdIUukKQimZmzVNz\nYcxKzYVxfur19mb2FzOba2b/MLP0qKe/BvZJXWGMM7P+ZvZkxv4mmNlZqX+/b2a/NLOXgJPNbB8z\ne9rM5pjZDDM7sHZ73P1Td58FbMj7hxeppaLGMpKK18bM5qX+/Z67DwbOAVa6ex8z2wZ42cymEUY9\nHezuq8xsZ+BVM3scuIowb0YPADPr38gx17n711Pb/gX4ibsvNLO+wG3AN+P+kCJNpYIgleTL9Bd5\nhgFANzM7KbW+HbAfYaC7sWZ2DFBNGAZ71yYc80HYPFrmkcAfw/A4AGzThP2J5I0KglQ6Ay529xpj\nNqVu++wC9HL3DWb2PtC6jvdvpOat19rbrEn9bAasqKMgiRQNZQhS6Z4BLkgNcYyZ7W9m7QhXCp+m\nisE3CEEvwGrCIIhpHwBdzGwbM9sOOLaug6TG0H/PzE5OHcfMrHt+PpJI0+gKQSrdXUBnYG5qqOOl\nhCd8JgNPmNlsYB7wNoC7Lzezl1OTz09198vN7CHgDWAh8FoDxxoK3G5mPwdaEqbofD1zAzP7f8Bs\nYFug2swuAbqU86QsUjz02KmIiAC6ZSQiIikqCCIiAqggiIhIigqCiIgAKggiIpKigiAiIoAKgoiI\npKggiIgIAP8fpA6wSCx2q8IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xb328a90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 权重初始化\n",
    "w = np.random.randn(3,1)\n",
    "\n",
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "第 0次更新，分类错误的点个数：23\n",
      "第 1次更新，分类错误的点个数：34\n",
      "第 2次更新，分类错误的点个数：56\n",
      "第 3次更新，分类错误的点个数：39\n",
      "第 4次更新，分类错误的点个数：36\n",
      "第 5次更新，分类错误的点个数：21\n",
      "第 6次更新，分类错误的点个数： 7\n",
      "第 7次更新，分类错误的点个数：23\n",
      "第 8次更新，分类错误的点个数：26\n",
      "第 9次更新，分类错误的点个数：21\n",
      "第10次更新，分类错误的点个数：23\n",
      "第11次更新，分类错误的点个数：18\n",
      "第12次更新，分类错误的点个数：22\n",
      "第13次更新，分类错误的点个数：15\n",
      "第14次更新，分类错误的点个数：18\n",
      "第15次更新，分类错误的点个数：15\n",
      "第16次更新，分类错误的点个数：17\n",
      "第17次更新，分类错误的点个数：11\n",
      "第18次更新，分类错误的点个数：16\n",
      "第19次更新，分类错误的点个数：12\n",
      "第20次更新，分类错误的点个数：16\n",
      "第21次更新，分类错误的点个数：11\n",
      "第22次更新，分类错误的点个数：16\n",
      "第23次更新，分类错误的点个数：11\n",
      "第24次更新，分类错误的点个数：16\n",
      "第25次更新，分类错误的点个数：11\n",
      "第26次更新，分类错误的点个数：14\n",
      "第27次更新，分类错误的点个数：10\n",
      "第28次更新，分类错误的点个数： 5\n",
      "第29次更新，分类错误的点个数： 8\n",
      "第30次更新，分类错误的点个数： 4\n",
      "第31次更新，分类错误的点个数： 7\n",
      "第32次更新，分类错误的点个数： 5\n",
      "第33次更新，分类错误的点个数： 5\n",
      "第34次更新，分类错误的点个数： 6\n",
      "第35次更新，分类错误的点个数：10\n",
      "第36次更新，分类错误的点个数：13\n",
      "第37次更新，分类错误的点个数：10\n",
      "第38次更新，分类错误的点个数：13\n",
      "第39次更新，分类错误的点个数：10\n",
      "第40次更新，分类错误的点个数：13\n",
      "第41次更新，分类错误的点个数：10\n",
      "第42次更新，分类错误的点个数：13\n",
      "第43次更新，分类错误的点个数：10\n",
      "第44次更新，分类错误的点个数：12\n",
      "第45次更新，分类错误的点个数：10\n",
      "第46次更新，分类错误的点个数：11\n",
      "第47次更新，分类错误的点个数：11\n",
      "第48次更新，分类错误的点个数：11\n",
      "第49次更新，分类错误的点个数： 6\n",
      "第50次更新，分类错误的点个数： 5\n",
      "第51次更新，分类错误的点个数： 5\n",
      "第52次更新，分类错误的点个数： 7\n",
      "第53次更新，分类错误的点个数：10\n",
      "第54次更新，分类错误的点个数：11\n",
      "第55次更新，分类错误的点个数： 4\n",
      "第56次更新，分类错误的点个数： 6\n",
      "第57次更新，分类错误的点个数： 8\n",
      "第58次更新，分类错误的点个数： 9\n",
      "第59次更新，分类错误的点个数： 4\n",
      "第60次更新，分类错误的点个数： 6\n",
      "第61次更新，分类错误的点个数： 6\n",
      "第62次更新，分类错误的点个数： 5\n",
      "第63次更新，分类错误的点个数： 4\n",
      "第64次更新，分类错误的点个数： 6\n",
      "第65次更新，分类错误的点个数： 8\n",
      "第66次更新，分类错误的点个数： 9\n",
      "第67次更新，分类错误的点个数： 4\n",
      "第68次更新，分类错误的点个数： 6\n",
      "第69次更新，分类错误的点个数： 5\n",
      "第70次更新，分类错误的点个数： 5\n",
      "第71次更新，分类错误的点个数： 4\n",
      "第72次更新，分类错误的点个数： 6\n",
      "第73次更新，分类错误的点个数： 8\n",
      "第74次更新，分类错误的点个数：10\n",
      "第75次更新，分类错误的点个数： 4\n",
      "第76次更新，分类错误的点个数： 6\n",
      "第77次更新，分类错误的点个数： 6\n",
      "第78次更新，分类错误的点个数： 8\n",
      "第79次更新，分类错误的点个数： 4\n",
      "第80次更新，分类错误的点个数： 7\n",
      "第81次更新，分类错误的点个数： 7\n",
      "第82次更新，分类错误的点个数： 9\n",
      "第83次更新，分类错误的点个数： 4\n",
      "第84次更新，分类错误的点个数： 6\n",
      "第85次更新，分类错误的点个数： 6\n",
      "第86次更新，分类错误的点个数： 8\n",
      "第87次更新，分类错误的点个数： 4\n",
      "第88次更新，分类错误的点个数： 6\n",
      "第89次更新，分类错误的点个数： 6\n",
      "第90次更新，分类错误的点个数： 8\n",
      "第91次更新，分类错误的点个数： 4\n",
      "第92次更新，分类错误的点个数： 6\n",
      "第93次更新，分类错误的点个数： 7\n",
      "第94次更新，分类错误的点个数： 8\n",
      "第95次更新，分类错误的点个数： 5\n",
      "第96次更新，分类错误的点个数： 5\n",
      "第97次更新，分类错误的点个数： 4\n",
      "第98次更新，分类错误的点个数： 6\n",
      "第99次更新，分类错误的点个数： 6\n",
      "第100次更新，分类错误的点个数： 8\n",
      "第101次更新，分类错误的点个数： 4\n",
      "第102次更新，分类错误的点个数： 6\n",
      "第103次更新，分类错误的点个数： 6\n",
      "第104次更新，分类错误的点个数： 8\n",
      "第105次更新，分类错误的点个数： 4\n",
      "第106次更新，分类错误的点个数： 5\n",
      "第107次更新，分类错误的点个数： 4\n",
      "第108次更新，分类错误的点个数： 6\n",
      "第109次更新，分类错误的点个数： 5\n",
      "第110次更新，分类错误的点个数： 7\n",
      "第111次更新，分类错误的点个数： 4\n",
      "第112次更新，分类错误的点个数： 6\n",
      "第113次更新，分类错误的点个数： 6\n",
      "第114次更新，分类错误的点个数： 8\n",
      "第115次更新，分类错误的点个数： 4\n",
      "第116次更新，分类错误的点个数： 6\n",
      "第117次更新，分类错误的点个数： 6\n",
      "第118次更新，分类错误的点个数： 8\n",
      "第119次更新，分类错误的点个数： 5\n",
      "第120次更新，分类错误的点个数： 5\n",
      "第121次更新，分类错误的点个数： 5\n",
      "第122次更新，分类错误的点个数： 7\n",
      "第123次更新，分类错误的点个数： 5\n",
      "第124次更新，分类错误的点个数： 5\n",
      "第125次更新，分类错误的点个数： 5\n",
      "第126次更新，分类错误的点个数： 4\n",
      "第127次更新，分类错误的点个数： 6\n",
      "第128次更新，分类错误的点个数： 8\n",
      "第129次更新，分类错误的点个数： 5\n",
      "第130次更新，分类错误的点个数： 5\n",
      "第131次更新，分类错误的点个数： 4\n",
      "第132次更新，分类错误的点个数： 6\n",
      "第133次更新，分类错误的点个数： 6\n",
      "第134次更新，分类错误的点个数： 8\n",
      "第135次更新，分类错误的点个数： 4\n",
      "第136次更新，分类错误的点个数： 5\n",
      "第137次更新，分类错误的点个数： 5\n",
      "第138次更新，分类错误的点个数： 5\n",
      "第139次更新，分类错误的点个数： 6\n",
      "第140次更新，分类错误的点个数：10\n",
      "第141次更新，分类错误的点个数： 4\n",
      "第142次更新，分类错误的点个数： 5\n",
      "第143次更新，分类错误的点个数： 5\n",
      "第144次更新，分类错误的点个数： 5\n",
      "第145次更新，分类错误的点个数： 5\n",
      "第146次更新，分类错误的点个数： 4\n",
      "第147次更新，分类错误的点个数： 6\n",
      "第148次更新，分类错误的点个数： 8\n",
      "第149次更新，分类错误的点个数： 5\n",
      "第150次更新，分类错误的点个数： 6\n",
      "第151次更新，分类错误的点个数： 5\n",
      "第152次更新，分类错误的点个数： 7\n",
      "第153次更新，分类错误的点个数： 5\n",
      "第154次更新，分类错误的点个数： 7\n",
      "第155次更新，分类错误的点个数： 5\n",
      "第156次更新，分类错误的点个数： 5\n",
      "第157次更新，分类错误的点个数： 5\n",
      "第158次更新，分类错误的点个数： 4\n",
      "第159次更新，分类错误的点个数： 6\n",
      "第160次更新，分类错误的点个数： 8\n",
      "第161次更新，分类错误的点个数： 5\n",
      "第162次更新，分类错误的点个数： 6\n",
      "第163次更新，分类错误的点个数： 5\n",
      "第164次更新，分类错误的点个数： 7\n",
      "第165次更新，分类错误的点个数： 5\n",
      "第166次更新，分类错误的点个数： 5\n",
      "第167次更新，分类错误的点个数： 5\n",
      "第168次更新，分类错误的点个数： 4\n",
      "第169次更新，分类错误的点个数： 6\n",
      "第170次更新，分类错误的点个数： 9\n",
      "第171次更新，分类错误的点个数： 5\n",
      "第172次更新，分类错误的点个数： 7\n",
      "第173次更新，分类错误的点个数： 5\n",
      "第174次更新，分类错误的点个数： 5\n",
      "第175次更新，分类错误的点个数： 5\n",
      "第176次更新，分类错误的点个数： 4\n",
      "第177次更新，分类错误的点个数： 6\n",
      "第178次更新，分类错误的点个数： 9\n",
      "第179次更新，分类错误的点个数： 5\n",
      "第180次更新，分类错误的点个数： 7\n",
      "第181次更新，分类错误的点个数： 5\n",
      "第182次更新，分类错误的点个数： 5\n",
      "第183次更新，分类错误的点个数： 5\n",
      "第184次更新，分类错误的点个数： 5\n",
      "第185次更新，分类错误的点个数： 5\n",
      "第186次更新，分类错误的点个数： 4\n",
      "第187次更新，分类错误的点个数： 6\n",
      "第188次更新，分类错误的点个数： 8\n",
      "第189次更新，分类错误的点个数： 5\n",
      "第190次更新，分类错误的点个数： 5\n",
      "第191次更新，分类错误的点个数： 5\n",
      "第192次更新，分类错误的点个数： 5\n",
      "第193次更新，分类错误的点个数： 5\n",
      "第194次更新，分类错误的点个数： 4\n",
      "第195次更新，分类错误的点个数： 6\n",
      "第196次更新，分类错误的点个数： 9\n",
      "第197次更新，分类错误的点个数： 5\n",
      "第198次更新，分类错误的点个数： 7\n",
      "第199次更新，分类错误的点个数： 5\n"
     ]
    }
   ],
   "source": [
    "for i in range(200):\n",
    "    s = np.dot(X, w)\n",
    "    y_pred = np.ones_like(y)\n",
    "    loc_n = np.where(s < 0)[0]\n",
    "    y_pred[loc_n] = -1\n",
    "    num_fault = len(np.where(y != y_pred)[0])\n",
    "    print('第%2d次更新，分类错误的点个数：%2d' % (i, num_fault))\n",
    "    if num_fault == 0:\n",
    "        break\n",
    "    else:\n",
    "        t = np.where(y != y_pred)[0][0]\n",
    "        w += y[t] * X[t, :].reshape((3,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "迭代完毕后，得到更新后的权重系数$w$ ，绘制此时的分类直线是什么样子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEKCAYAAAASByJ7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3XuYFNWdN/Dvb27AAKISVBBnxgsq\nwwg4gKhRB5UAmrxGYrwtCavRjAGVgVVXfUwC+yS4a/QRIUYENdnXZTbRVSTZRIRgdEXFxAFRUQx4\nQ2dNXgHlIjeZ6d/7x+mevkzPzOnpqj7V1d/P8/QzVE911enSqV9VfeucElUFERFRkesGEBFRMLAg\nEBERABYEIiKKYkEgIiIALAhERBTFgkBERABYEIiIKIoFgYiIALAgEBFRVInrBmTiK1/5ilZVVblu\nBhFRXlm7du02VR3Q1Xx5VRCqqqrQ1NTkuhlERHlFRLbYzMdLRkREBIAFgYiIolgQiIgIQJ5lCOkc\nPHgQzc3N2L9/v+um5KWePXti8ODBKC0tdd0UInIs7wtCc3Mz+vbti6qqKoiI6+bkFVXF9u3b0dzc\njGOPPdZ1c4jIsby/ZLR//37079+fxaAbRAT9+/fn2RURAQhBQQDAYpAFbjsi7zQ2AlVVQFGR+dnY\n6LpFmcn7S0ZEREHQ2AjU1wN795rpLVvMNABMmeKuXZkIxRmCa8XFxRg5ciRqampw6aWXYm/s/4gM\nXHvttXj77bcBAHfeeWfS784880xP2klE/rnjjngxiNm717yfL1gQPNCrVy+sX78eGzZsQFlZGR58\n8MGMl/Hwww+juroaQPuC8PLLL3vSTiLyz0cfZfZ+EBVcQfD7Gt/ZZ5+Nd999FwBw7733oqamBjU1\nNbjvvvsAAHv27MHXv/51jBgxAjU1NXjssccAAOPGjUNTUxNuu+027Nu3DyNHjsSU6Hlmnz59AACX\nX345nn766bZ1XXXVVXjyySfR2tqKW265BWPGjMHw4cOxaNEib78UEXWpoiKz9wNJVfPmNWrUKE31\n9ttvt3uvI0uWqJaXqwLxV3m5eT8bvXv3VlXVgwcP6kUXXaQPPPCANjU1aU1NjX7xxRe6e/dura6u\n1nXr1ukTTzyh1157bdtnd+zYoaqqdXV1+uqrryYtL3X5S5cu1alTp6qq6oEDB3Tw4MG6d+9eXbRo\nkf7kJz9RVdX9+/frqFGj9P3337dufybbkIjS82v/4gUATWqxjy2oMwS/rvHFjuhHjx6NiooKXHPN\nNXjxxRcxefJk9O7dG3369MG3vvUtrF69GqeccgpWrVqFW2+9FatXr0a/fv2s13PBBRfgT3/6Ew4c\nOIDly5fjnHPOQa9evbBy5Uo8+uijGDlyJMaOHYvt27dj8+bN2X0pIsrIlCnA4sVAZSUgYn4uXpw/\ngTJQYHcZ+XWNL5YhJDJFub0TTzwRa9euxdNPP43bb78dEyZMwI9//GOr9fTs2RPjxo3DihUr8Nhj\nj+HKK69sW9fPf/5zTJw4MbsvQkRZmTIlvwpAqoI6Q8jlNb5zzjkHy5Ytw969e7Fnzx489dRTOPvs\ns/HJJ5+gvLwc3/nOd3DzzTdj3bp17T5bWlqKgwcPpl3uFVdcgV/96ldYvXp1WwGYOHEiFi5c2PaZ\nTZs2Yc+ePd5/KSIKtYI6Q5g7N/k+YQAoLzfve622thZXXXUVTjvtNADmttJTTz0VK1aswC233IKi\noiKUlpZi4cKF7T5bX1+P4cOHo7a2Fo0pqfeECRMwdepUXHTRRSgrK2tb9ocffoja2lqoKgYMGIBl\ny5Z5/6WIKNSko0sbQTR69GhNfUDOxo0bMXToUOtlNDaazOCjj8yZwdy5+X2K54VMtyERpTd9uskN\nWluB4mJzAPrAA65bBYjIWlUd3dV8BXWGAOT/NT4iCqbp04HEE/7W1vh0EIqCjYLKEIiI/LJ4cWbv\nBxELAhGRB1pbM3s/iFgQiIg8UFyc2ftBxIJARNQFmyFvYiOb2r4fRAUXKhMRZcJ2WOtYcBzEu4xs\nOTtDEJFjROQ5EdkoIm+JSIOrtmRLRHDTTTe1Td9zzz2YM2eO5+vhsNhEuZfJkDcPPAC0tJiRjFpa\n8qsYAG4vGbUAuElVhwI4HcD1IlLt+1pT+1140A+jR48eWLp0KbZt25b1sjrDYbGJci8Mw1rbclYQ\nVPVvqrou+u/dADYCONrXlc6ZA8yaFS8CqmY6y6P5kpIS1NfXY968ee1+t3XrVlxyySUYM2YMxowZ\ng5deeqnt/a997Wuora3Fddddh8rKyraCcvHFF2PUqFEYNmwYFkfvWeOw2ERuhGJYa1s2Q6L6/QJQ\nBeAjAIek+V09gCYATRUVFe2GdbUeujkSUW1oMGPSNjSkn+6m3r17686dO7WyslJ37Nihd999t86e\nPVtVVa+88kpdvXq1qqpu2bJFTz75ZFVVvf766/XOO+9UVdXly5crAN26dauqqm7fvl1VVffu3avD\nhg3Tbdu2ta0ndb2q2Q+LzeGviToW5GGtbcFy+GvnobKI9AHwJICZqror9fequhjAYsAMXZHFioDY\nEfz8+eYFAA0N5v0sHzZ/yCGHYOrUqViwYAF69erV9v6qVavaHo0JALt27cLu3bvx4osv4qmnngIA\nTJo0CYcddljbPAsWLGj73ccff4zNmzejf//+Ha77ggsuwIwZM3DgwAE888wzScNiv/HGG3jiiScA\nADt37sTmzZtx7LHHZvVdiQpJLDguhCFvnBYEESmFKQaNqro0Bys0O/9YMQA8KQYxM2fORG1tLa6+\n+uq29yKRCNasWZNUJICOh8d+/vnnsWrVKqxZswbl5eUYN24c9u/f3+l6OSw2kb8KZcgbl3cZCYBH\nAGxU1XtzstJYZpAoMVPI0uGHH47LLrsMjzzySNt7EyZMwP333982HXtuwllnnYXHH38cALBy5Up8\n/vnnAMxR/GGHHYby8nK88847eOWVV9o+y2GxichPLu8y+iqA7wI4T0TWR18X+ra2WDGYP99cJopE\nzM/58z0tCjfddFPS3UYLFixAU1MThg8fjurqajz44IMAgNmzZ2PlypWora3F8uXLMXDgQPTt2xeT\nJk1CS0sLhg8fjh/96Ec4/fTT25YVGxZ7SppDlQkTJuCFF17A+PHjk4bFrq6uRm1tLWpqanDdddeh\npaXFk+9JROFTWMNfz5kD7NgRv0wUKxKHHpr1nUaZOnDgAIqLi1FSUoI1a9Zg2rRp7Z66lisc/poo\nmLwarp/DX6czZ44pArHMIJYpeJQhZOKjjz7CZZddhkgkgrKyMjz00EM5bwMRBZdtD2kvFVZBANrv\n/B0UAwAYMmQIXnvtNSfrJqLg66yHtF8FIRSD2+XTZa+g4bYjCiYXPaTzviD07NkT27dv546tG1QV\n27dvR8+ePV03hYhSuOghnfeXjAYPHozm5mZs3brVdVPyUs+ePTF48GDXzSCyUkjPRJ87NzlDAIDy\ncvO+X/K+IJSWlrLnLVEBcBGyuuSih3Te33ZKRIWhqsoUgVSVlcCHH+a6NfnF9rbTvM8QiKgwFNIw\n1K6wIBBRXiioYagdYUEgorwwd64JVRP5HbIWGhYEIsoLU6aY5xVXVpr+pJWVZjqMgbIreX+XEREV\njkIZhtoVniEQEREAFgQiIopiQSAi5xobTT+DoiLzs7ExWMsL+nq9wgyBiJzyugeyqx7NYehJzZ7K\nROSU1z2QXfVoDnJPavZUJqK84HUPZFc9msPQk5oFgYic8roHsqsezWHoSc2CQNSJfA8J88HcuUBp\nafJ7paXd74HsqkdzGHpSsyAQdSAWEm7ZYh7FHQsJWRS85+WTbV31aA5DT2qGykQdCHJIGCbczv5j\nqEyUpTCEhPmA2zk4WBCIOhCGkDAfcDsHBwsCUQfCEBLGBLnnbpi2c95T1bx5jRo1SolyackS1cpK\nVRHzc8kS1y3K3JIlquXlqiYaN6/ycv+/SybrDcN2DjIATWqxj2WoTBRy7LlLDJWJCAB77pI9FgSi\nkGPPXbLFgkCUQy6Gec6HnrvsER4QNkFDUF4MlSmfeR3u5kNoa7NeV6F3IQFDZaJgCcswz14Ly/cI\nsrwIlUXklyLyqYhscNkOolwIyzDPXgvL9wgD1xnCvwOY5LgNRDkRlmGevRaW7xEGTguCqr4A4DOX\nbSDKFa/D3blzTQibqKgou7DYRbjLnsrB4foMgahgeD088ksvAZFI8nuRiHm/O1wN9x2GYaPDwnmo\nLCJVAH6vqjUd/L4eQD0AVFRUjNqSLn0iKkAlJUBra/v3i4uBlpbMl8dwN7zyIlS2oaqLVXW0qo4e\nMGCA6+YQBUa6YtDZ+11huEuBLwhElF5xcWbvd4XhLrm+7fTXANYAOElEmkXkGpftofzmRyA6fbq5\nNCNifk6fnt18Xqqvz+z9rjDcJee9jzN5sacydcSP3q7TpiUvL/aaNq1783ltyRLV4uLkdRYXZ/ed\nOQx1OIE9lamQ+BGI2oa2Xoe7thgCk63QhMpENvwIRG1DW6/DXVsMgclrLAgUCn4Eorahrdfhri2G\nwOQ1FgQKBT8CUdvQNpNw18vgm8NLk+dsgoagvBgqU2f8CESnTYsHt8XF6YPiJUtUS0qSw92Skvbr\n9yP45vDSZAMMlYlywzbc5bONyRWGykQ5Yhvu8tnGFHQsCERZsg13+WxjCjoWBCo4XgestuEun21M\ngWcTNATlxVCZsuVXwGobaPPZxuQCGCoTtceAtWPcNuHFUJkoDQasHeO2oQ4LgoicIiKviMjHIrJY\nRA5L+N1fctM8Im8xYO0Ytw11doawEMAcAKcA2ATgRRE5Pvq7Up/bReQLv4JdF8Nfe43DX1NnBaGP\nqj6jqjtU9R4ANwB4RkROB5A/wQNRAj+e3zt9OrBwYXwwu9ZWM51vRYHPNqYOQ2UReR3AOaq6M+G9\n4QCeBHC4qvbPTRPjGCpTELka/prIlheh8l0Ahia+oapvADgfwNLsmkcUHq6GvybyWklHv1DV/+zg\n/Y8AfN+3FhHlmeLijs8QiPIJbzslypKr4a/9WB4Vtg7PEIjIzgMPmJ+LF5szheJiUwxi78c0Npr3\n9+4101u2xItGd4Jbr5dHKfbtAzZuBDZsAA45BLj4Ytct8h17KhPliNc9gdmz2CMHDwLvvmt2/G++\naX5u2AC89x4QiZh5zjsPePZZt+3Mgm2o3OUZgoicCNMn4UhVrYneaXSRqv7Ug3YSFQyvewKzZ3GG\nIhGzcVJ3/O+8A3z5pZmnqAg44QRg+HDgH/4BqKkxrxNOcNv2HLG5ZPQQgFsALALMnUYi8p8AWBAo\nN1TNjfEdTeeJior0R/Td7Qns9fJCQxX49NP2O/633gK++CI+3zHHAKecAkyaFN/xn3wy0KuXu7Y7\nZhMql6tq6lAVvLuaOjR+vNlfx17jx6efzyoQnTMHmDXL/JED5uesWeb9HPD6GchlZcnvlZV1vyew\nHz2L8y6k3rkTePllYNEi4MYbgXPPBY44AjjqKPM/3qxZwG9/azbM1Veb+V56Cdixw5wt/OEPwF13\nAd/9LnDqqQVdDAC7M4Rt0SErFABE5NsA/uZrqyhvjR/f/lLrs8+a91etir9nFYiqmj/c+fPN9Lx5\n5g98/nygocH3MwU/QtvUyC6bCC/WhjvuMPu2igpTDLrbtkCH1Pv2mUs7iUf8GzYAH38cn6dPH3OU\nf/HF8SP+mhpTIPLwjNKFLkNlETkOwGIAZwL4HMAHAKaoapqTVX8xVA6+zv7uEv9Xsw5EY2cEsaIA\nmGIwb57vf+SFFgIHon0tLekD3nffjQe8ZWXA0KHJO/2aGlMRi3gnfTq2oXKnBUFEigB8W1UfF5He\nAIpUdbeH7cwIC0Lw2RaEoqL0R8ci8b/7pA8m/qFHIjk54suojQ6W57Wctk81fcC7cWP7gDd1x3/C\nCUApx9fMhCd3GalqRERuAPC4qu7xrHVU8KwD0dgZQqJZs3JyhpAXIbCHgbtvIXVHAe/uhGPLY44x\nO/sJE+I7/qFDC/6afs519Ug1AD8CcDOAYwAcHnvZPI7N6xcfoemWzWMYzz8/+RGMsdf557dfVpeP\na4xEVBsaVAF9pG+DCiL6SF8zrQ0N5vc+WrJEtbQ0uY2lpd1/pKTnj6icPTt5O8S21+zZbtq3c6fq\nSy+pLlqkeuONqueeqzpgQPIC+/dXHTdO9YYbVB98UPXFF1U//7xb7SV7sHyEpk1B+CDN632bhXv9\nYkFwJ5OdRWpRSC0GicvsqsC8Pnm23l/SoEAkuryI3l/SoK9Pnu3ht+u4fWVlyd+lrCy7Zwx79kzl\nhGLZVhRSp/1q3759quvWqT76qOo//7PqhReqVlQkb6jevVXHjlW95hrVefNU//hH1b/9zfciTunZ\nFgT2VCYrrgJHs14FkHgZRFFZKb4HnYEIWTujPgfuLS2mt27qnT2bN8dDhdLS9AFvZSUD3gDxJFSO\nLmhquvdV9dFutq3bWBDccRWIugxigx4CA/AmcFc1t2+m7vg3bgQOHDDziKQPeIcMYcCbBzwbugLA\nmIR/94R5HsI6AFkXBBGZBGA+gGIAD6vqv2W7TPKHq16xLnvjmnW3PzupqAjIPe2xM4REXQXuW7e2\n3/Fv2JAc8A4ebHb248cDw4aZYRxOPrl9LzgKH5vrSokvAP0A/C7Tz6VZTjGA9wAcB6AMwOsAqjv7\nDDMEf9hcN/Y8EM2gbbbr9ez6fNTrk2frgqLk/GJBUW7yiy51lSHs2KH68suqixerzpihet55qkcc\nkbwhDz9cta5O9frrVRcuVL36atUf/MCzkJqCA16Fyu0+AJQC2Jjp59Is5wwAKxKmbwdwe2efYUHw\nnssdbiZtzHnBikR040Szg50HUxTmwUxvnOj/HU5WZs82O/N161T/4z9MwFtVpdqvX/uA97TTVL/3\nPdV771VdubJ9wOtTSE3BYFsQbDKE/0Z02AqYsY+qAfyXqt6azZlJdAiMSap6bXT6uwDGquoNHX2G\nGYL3Ah+cWvLje1RVKmZ+NAszEQ9t70MD7quYhw+35PiyUWtrvAdvasAbe1xbaam5tJN4jf+UU+wD\nXr9DanLGy1C5LmGyBcAWVW3Osn0QkUsBTEwpCKep6o0p89UDqAeAioqKUVvS/dVTt+VFcGrBj+9h\nlqnQhDEgBRGIiH/bJhbwpu743347OeA9/vj2O34vAl4vQmoKHC9D5QtTzwZE5K5szxAANMN0dosZ\nDOCT1JlUdTHMWEoYPXp0/twjmyecDqGswe5lW3GMOUNINA+zcN8x85AcNHfT1q3td/wbNgC7dsXn\nOfpos8M/7zyz04/14M004LXZ1rEzhEQ56hVOAdHVNSUA69K894bN9agullsC4H0AxyIeKg/r7DPM\nELznKizOpJet1xmCVRbiZYawa5fqmjWqDz1kvuP556seeWRyYw87TPWcc1SnTzcB7+rVqp99Zr+O\nzthsa2YIoYZsQ2UA0wC8CWAPgDcSXh8AWGKz8C5XDlwIYBPM3UZ3dDU/C4I/ch4WZ7Dz8XpHn1EB\nnD1bN05s0MqKiFlmRbRIdHTXzf79quvXm4XddpvqN75hQt7UlY0ZY+7oiQW8n3zi3w43kx29x0Nh\nUHDYFoQOMwQR6QfgMAD/CuC2hF/tVtXPvDpDyQRD5RCxDDCdD0Gd7lJLJGJ68KZe6tm0KR7wlpTE\nA97YpZ6amvjTZ3Ipk7DYw8t4FByehcoJCzwCpmMaAEBVc/7kVhaEkLEIMJ0OQa0KNDenD3j3749/\n8Ljj2u/4hwxp/3g0lyy2NYWXZ6GyiPwfAPcCGATgUwCVADYCGJZtI6mAWQaYGYXFkUj7nV7K0XhH\nyxtx9Dbg+TQB786d8ZkGDTI7++uvTx6iuXdviy/sE5sjestt7fl6Kf90dU0JJuztD+C16PS5ABbb\nXI/y+sUMIST8yBDq6lRHjlRtbTXTra1muq4uabbfPLRLz+nxin4PD+u9mKkrMV7/jjQB79lnq44e\nbYZw/p//Ud2+PXjX1F2Fxcwa8g4sMwSb204Pqup2ESkSkSJVfU5E7vKtQlH4iQCHHpp8HXvePPO7\nQw9NOtK0em5wJGKO5NevB0aNAtauNQ9Mf+MNEw7cfrt5IMubb+LyDz/E5dGP7UUvbC4bhi9OvxBH\nfjPhnv6BA80Msevuw4cDZ5+d0+c5d0nV7nnTGWxrT9dL+amrigFgFYA+AO4H8GuYwehetqk2Xr94\nhhAyqUen3TlabWlR3bRJ9YknVI86KvlIP/YqKVEdNkz1iitUf/pT1WXLVN99N3420Vn7YkfTsVeQ\nbsHMpH1ebOvurJcCAR4OXdEbwD6YYSumwAxu16iq2/0rU+kxVC5gqsD//m/6gHffvvSfaWw0R/cn\nntj9gFcDHsa6al/Qtwsl8SxUVtU9IlIJYIiq/l8RKYcZqZTIH9u3J+/0Y8M1Jwa8Aweau3qmTTOX\neaqrgWuvNfPF/OxnwLp13b/NUx2GsTbz+dE+G67WS/7r6hQCwPcBvArgvej0EADP2px+eP3iJaOQ\n2b1b9c9/Vn3kEdVZs1THj29/2efQQ03AO22a6i9+YQLebduSl9PaGu/5O2KEmR4xwkwfeWTXl4bS\ncRnGBrlnMXs05yV4GCpfD+A0AH+OFpDN0T4JRHa+/BL461/bH/F/8EF8nl69zMNYJk1KHrRt0KCu\njzpFTEcwAKirM9N1dcDrr5v3u3PU6iqMtZ3P6/bZcrVeygmbDOHPqjpWRF5T1VNFpARmfKPhuWli\nHDOEgGttNTv51Ms9mzaZ5/MCZgd90kntH8V47LFAcRZXIlWBmTOBBQvi782YAdx3X3Y7KdtLPLbL\nsukxbDuf1+3LhKv1Urd4Ofz1zwDsADAVwI0ApgN4W1Xv8KKhmWBBCAhV4JNP2u/4UwPeWA/exNdJ\nJ/nXgzcfgk7bNubDd6G84eXw17cBuAZmoLvrADwN4OHsmkdZ8frorLPlffZZ+0s9GzaYyxoxAwea\nnf0PfhDf8VdXA336dL9NiSx6IGcUdPpxdOtlCMzQllzpKFwAUGETQuTylW2o7OoRkJ7yupdobHm7\nd6v+5S8m4D31VNXjjlMdODA54C0rUz3rLPPc3Z//XPWSS1RvucWjL9YBmx7Irkf09DIEZmhLPoAH\nofIyALUAICJPquolOahPvmlsBOrrgb17zfSWLWYaSOn1GmS2gWNnvvzSXNOPHfH/5jfA++8nX68G\ngCOOAC64wBztDxsGPPEE8Mtfmp7AsfU++aS/vVPT9UAeNcpMjxwZP1OwDTq92H6pvA6BGdqSSx1V\nCkTHLkr9t8tXNmcIlZXJB7uxV2Vltxfphm0v0dZW0xt32TLTO/fyy01v3ZKS+OeKi1Wrq1WHDEle\n3o03pu/Z6qJ3auyMIHG9iWcMqW3sbDr2ntffw48ew172LKaCBw+eh7BOVWtT/+1SNqFyWJ4dDKB9\n4NjcbMbqSbzO//bb8dMhwNzFk/oM3hNPBHr0CH7QGYkk34HU2prdMwX8+B6utg2RBS9C5REisgvm\n4bG9ov9GdFpV9RAP2pkzTp8d7JVYwDtnTvL7gwfH/33UUWaHX19vLvUMHx4PeNNdEtGAB52RiLlM\nlCh2+ag7RSHT75G6zbLZhpm206sezUS2bE4jgvLK5pKRs2cHd8cXX6i++qrqr36l+k//pDphguqg\nQcmNLytT/epXVWtqzPQll6h++ml8GWEIOhMvF8UuE6VOZyLT7xH04aU5DDVZgoc9lUPBahjlXDt4\nMDngjd3S+f778etbPXuaI/zx482R/+uvm8s8Dz0Uvw42a5YJHAcMMJ/RkASdRUVAv34mQI6dEcSC\n5X79Mj9DyOR7eL0Nbdmu13Y+okzYVI2gvPJ2LKPWVtX33lP97W9NwHvFFebIvrQ0fsRfXKw6dKjq\npZeq/su/qC5daoZ1bmlpvzyvw9OgB52pZwLdGZsoUSbfN8jDS3MYarIEr4a/DpLA91RWBf7+9/ZH\n/G+9lRzwVlUlh7uxHrw9enjfHgad2XG1DW3Xy//GZMHLnsqUzueftz2FK2kIh88+i89z5JFmZ//9\n78d3/NXVQN++/rcvdikpEXu7GqmXUzq6vJLJNrRdpm37ghz0U3jZnEYE5eXkktGePapNTSbgvekm\n1YkTVY8+Ovk0/ZBDVM88U7W+3vTgfe655IA319jbtWO2Qayr3s9BD/opL4GhcoYOHgQ2b25/uee9\n9+IBb48e5gj/vPPiR/w1Nea2zyAdkbG3a3qaQRDrqvdz0IN+CjVmCM89Z/6o3nnHFAXAdIIaMiT5\nGn9NDXD88dkN0ZxrXl7GCIvYZRaboaVj89v2Q7Bdpm072Q+BPOLZ8NdB4ktBWL8e+OEPk3f8J51k\nbvekcPIjiGW4SwHGUNnWyJHA73/vuhVu2R5l2gxD7cd6veRHEMtwl0Iii79mCoU5c8zOK3amGNu5\npQ6PMW6c6RAWG/gpNqTEuHH+rtdLiZd2GhrMd2hoMNOJbXG9TCJHWBAKWWIgGtt5xXZuO3bEd2ap\nw1DHisH69eb9TEcHtF2v1zoKYhsauh/E+rFMIkeYIRQ620A0sQjEJA4p4dd6/eDHpSqGuxRgDJXJ\nnm0gmg/DUBNRO7YFgZeMCp0qMHNm8nszZ7a/bBOJALUpj8SorU1/uSj1s+kOOjoKYvPoAIUobJwU\nBBG5VETeEpGIiHRZtcgnqsAZZwALFgAzZpid+4wZZvqMM5IzhEGDzEirI0aYM4MRI8z0oEHJRcEm\nLGYQSxRIrs4QNgD4FoAXHK2fMiEClETvUK6rM9N1dWa6pCR9r93OwmIGsUTBZDO+hV8vAM8DGG07\nf94Ofx1kkYjqjBnJYzPNmJF+qGXb+VwMGU1EHUI+DH8tIs8DuFlVrZJihso+sQ13vZ6PiHLCeags\nIqtEZEOa1zczXE69iDSJSNPWrVv9am44pQa+HQXANuFuJvPZhNSZsAmpM5mPiNKzOY3w6wVeMvJP\nXV3yc4djzyOuq4vP4/VQy5GI6tixyZeTYpeZxo7l84WJHIHlJSPedhpGtj2LbcNdVyGwWobUtvMR\nUedsqobXLwCTATQDOADg/wFYYfM5niFkIHZGkBjuJp4xJPLymcq24bMtPl+YKGvIh1A5UwyVM+R1\nz2Jb6nGobLs8r9dLFBLOQ2X9oVgjAAAIVElEQVRKkFp0c1GEY5eJEiWOVtpZe7Jpn3ocKscu/yTK\nJvQmog6xIPjNxTDPiZnByJHmzGDkyORMwY/2qWXP50yWZ9Oj2XY+IuoUH5Djp8SwE8j+ebu2ioqA\nfv2SRyNdu9YUg3794pdVXLXPFp8vTJRbNkFDUF55GSq7DDtTA+SOAmUv2+d1qBxbZmfTmc5HVGDA\nUDlAgh52ugqBiSgnGCoHhcuw0yYs9rp9DHeJ8hYLgp9chp0uhqFmuEuU1xgq+8lV2GkbFnvdPoa7\nRHmNGUIupN6tk4u7dxKP1mM6emax1+1z8X2JqEN8pjIx3CUiAAyVyesew0QUeiwIYeR1j2EiKggs\nCEREBIAFIZxEgDVr4mcFRUXxs4U1a5gjEFFaDJXDjKEyEYGhMmXSY9jF8NxEFDgsCGGUSY9hF8Nz\nE1EgsadyGNn2GA768NdElFPMEMLMpsdwJj2aiSgvsacy2WP4TBRqDJWzUUghK8NnIopiQUhVSCEr\nw2ciSsCCkCgxZI3t/GI7zB07wndE3FH43NDQcfhcCNuFqEAxQ0hViCErw2eiUGOonA2GrOlxuxDl\nJYbK3cVnAqfH7UIUeiwIifhM4PS4XYgKAnsqJ+IzgdPjdiEqCMwQ0uEzgdPjdiHKS8wQspG6k+NO\nz+B2IQq1wioItj1t2SOXiApQ4RQE25627JFLRAXKSUEQkbtF5B0ReUNEnhKRQ31doW1PW/bIJaIC\n5iRUFpEJAP6kqi0ichcAqOqtXX0uq1DZtqcte+QSUcjkTU9lEZkM4NuqOqWrebO+y8i2py175BJR\niOTTXUbfA7Dc97XY9rRlj1wiKlC+FQQRWSUiG9K8vpkwzx0AWgA0drKcehFpEpGmrVu3dq8xtj1t\n2SOXiAqYbz2VVXV8Z78XkX8E8A0A52sn161UdTGAxYC5ZNStxtj2tGWPXCIqYK5C5UkA7gVQp6rW\nh/2eZAg2PW3ZI5eIQiToGcL9APoC+KOIrBeRB3OyVtuetuyRS0QFyMngdqp6gov1EhFRx4JwlxER\nEQUACwIREQFgQSAioigWBCIiAsCCkB0Ok01EIcKC0F0cJpuIQoYFoTs4TDYRhZCTfgh5L3FIi/nz\n40Nlc5hsIspjzoe/zkTWQ1d4jcNkE1EeCPrQFfmPw2QTUciwIHQHh8kmohBihtAdHCabiEKIGUI2\nOEw2EeUBZgi5wGGyiShEWBCIiAgACwIREUWxIBAREQAWBCIiimJBICIiACwIREQUlVf9EERkK4At\nPiz6KwC2+bDcfMRtYXA7GNwORr5vh0pVHdDVTHlVEPwiIk02nTYKAbeFwe1gcDsYhbIdeMmIiIgA\nsCAQEVEUC4Kx2HUDAoTbwuB2MLgdjILYDswQiIgIAM8QiIgoigUhSkTuFpF3ROQNEXlKRA513SYX\nRORSEXlLRCIiEvq7KlKJyCQR+auIvCsit7lujysi8ksR+VRENrhui0sicoyIPCciG6N/Fw2u2+Qn\nFoS4PwKoUdXhADYBuN1xe1zZAOBbAF5w3ZBcE5FiAL8AcAGAagBXiki121Y58+8AJrluRAC0ALhJ\nVYcCOB3A9WH+f4IFIUpVV6pqS3TyFQCDXbbHFVXdqKp/dd0OR04D8K6qvq+qXwL4DYBvOm6TE6r6\nAoDPXLfDNVX9m6qui/57N4CNAI522yr/sCCk9z0Ay103gnLuaAAfJ0w3I8R//JQZEakCcCqAP7tt\niX8K6pnKIrIKwFFpfnWHqv42Os8dMKeJjblsWy7ZbIcCle6Rd7wNjyAifQA8CWCmqu5y3R6/FFRB\nUNXxnf1eRP4RwDcAnK8hvh+3q+1QwJoBHJMwPRjAJ47aQgEhIqUwxaBRVZe6bo+feMkoSkQmAbgV\nwEWqutd1e8iJVwEMEZFjRaQMwBUAfue4TeSQiAiARwBsVNV7XbfHbywIcfcD6AvgjyKyXkQedN0g\nF0Rksog0AzgDwB9EZIXrNuVK9KaCGwCsgAkPH1fVt9y2yg0R+TWANQBOEpFmEbnGdZsc+SqA7wI4\nL7pfWC8iF7pulF/YU5mIiADwDIGIiKJYEIiICAALAhERRbEgEBERABYEIiKKYkGggiEirQm3Dq6P\nDkWQ6TIOFZHp3reubfkni8gaETkgIjf7tR6idHjbKRUMEflCVftkuYwqAL9X1ZoMP1esqq0W8x0B\noBLAxQA+V9V7utNOou7gGQIVNBEpjj4L49XoszCui77fR0SeFZF1IvKmiMRGPf03AMdHzzDuFpFx\nIvL7hOXdLyJXRf/9oYj8WEReBHCpiBwvIs+IyFoRWS0iJ6e2R1U/VdVXARz0/csTpSiosYyo4PUS\nkfXRf3+gqpMBXANgp6qOEZEeAF4SkZUwo55OVtVdIvIVAK+IyO8A3Abz3IyRACAi47pY535VPSs6\n77MAfqCqm0VkLIAHAJzn9Zck6i4WBCok+2I78gQTAAwXkW9Hp/sBGAIz0N2dInIOgAjMMNhHdmOd\njwFto2WeCeC/zPA4AIAe3VgekW9YEKjQCYAbVTVpzKboZZ8BAEap6kER+RBAzzSfb0HypdfUefZE\nfxYB2JGmIBEFBjMEKnQrAEyLDnEMETlRRHrDnCl8Gi0G58IEvQCwG2YQxJgtAKpFpIeI9ANwfrqV\nRMfQ/0BELo2uR0RkhD9fiah7eIZAhe5hAFUA1kWHOt4Kc4dPI4D/FpEmAOsBvAMAqrpdRF6KPnx+\nuareIiKPA3gDwGYAr3WyrikAForIDwGUwjyi8/XEGUTkKABNAA4BEBGRmQCqw/xQFgoO3nZKREQA\neMmIiIiiWBCIiAgACwIREUWxIBAREQAWBCIiimJBICIiACwIREQUxYJAREQAgP8P7/wA1pY3oIkA\nAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0xa185278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 直线第一个坐标（x1，y1）\n",
    "x1 = -2\n",
    "y1 = -1 / w[2] * (w[0] * 1 + w[1] * x1)\n",
    "# 直线第二个坐标（x2，y2）\n",
    "x2 = 2\n",
    "y2 = -1 / w[2] * (w[0] * 1 + w[1] * x2)\n",
    "# 作图\n",
    "plt.scatter(X[:50, 1], X[:50, 2], color='blue', marker='o', label='Positive')\n",
    "plt.scatter(X[50:, 1], X[50:, 2], color='red', marker='x', label='Negative')\n",
    "plt.plot([x1,x2], [y1,y2],'r')\n",
    "plt.xlabel('Feature 1')\n",
    "plt.ylabel('Feature 2')\n",
    "plt.legend(loc = 'upper left')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算分类正确率："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy: 0.95\n"
     ]
    }
   ],
   "source": [
    "s = np.dot(X, w)\n",
    "y_pred = np.ones_like(y)\n",
    "loc_n = np.where(s < 0)[0]\n",
    "y_pred[loc_n] = -1\n",
    "accuracy = len(np.where(y == y_pred)[0]) / len(y)\n",
    "print('accuracy: %.2f' % accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PCA是机器学习最简单的算法之一。PCA处理线性可分问题，优化的PCA解决线性不可分的问题。实际验证表明，一般的PCA处理线性可分及线性不可分问题都有不错的表现，即一般能得到最佳的分类直线。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
